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' The handbook is designed to help State Planning 
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performance monitoring systems in order to^ monitor the 
implementation, operation, and results of the criminal justice 
projects they support. Such monitoring is required by the Law 
Enforcement Assistiince Administration (LlIAAl . guidelines. The 
suggested procedures were obtained by telephone survey, review of* 
materials, and visits to 55 SPAs and. repres|ent those practices which 
appealed most useful in meeting the LEAA sidelines. A chapter \ 
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the required monitoring, and describe^/its relationship to management 
and other types of evaluation. In another chapter the major problems 
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four chapters detail hon the manager can use the monitoring system to 
accomplish tXese four identified tasks: establishing ifgreement on 
information required by the user, establishing agreement with the 
subgrantee (unit of local or State government tec eiting Federal funds 
from the SPA for a criminal justice ptoiect) on what will be 
monitored, esta^blishing the information flow, and assuring the 
utilization of monitoring information. App^id:^ material (79 pages) 
includes detailed examples (identified by State! of procedures and 
materials used in monitoring criminal justice programs*. 
(Author/MS) 
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FOREWORD 



Sound program decisions require accurate and meaningful information. Con* 
gress recognized this need and in the Crime Control Act of 1973 mandated a major 
evaluation of the impact of Federal assistance on the criminal justice system* For 
both the Law Enforcement Assistance Administration and the State Planning 
Agencies, increasing requests for continuation funding intensify this necessity for 
solid and precise information on program performance* 

This study analyzes effective systems for monitoring both the progress and 
performance of state and local criminal justice programs. It is designed to help 
agencies plan improvements in their monitoring techniques. The Appendix, which 
contains detailed monitoring materials and forms, may be particularly useful in 
developing specific procedures* 

Gerald M. Caflan 
Director 

National Institute of Law Enforcement 
and Criminal Justice 
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K INTRODUaiON 



•New Lay Enforcement Assistance Administration 
(LEAA) evaluation guidelines for State Planning 
Agencies (SPAs) require that each SPA monitor 
the implementation, operation and results of the 
projects it supports. Even before LEAA acted, sev- 
eral SPAs had recognized the need for these 
management activities by initiating new monitoring 
systems. Others have since begun to develop such 
systems. Nevertheless, the Birrent monitoring capa- 
bility of most states remains quite limited. 

This handbook is designed to help SPAs to de- 
velop or improve performance monitoring systems. 
It IS aimed specifically at those persons responsible 
for developing and operating a monitoring system. 

The suggested procedures presented here are not 
meant to be a rigid format for monitoring system 
managers to follow. They are culled from the prac- 
tices employed by the 55 SPAs and represent those 
that appear most useful in meeting the new LEAA 
guidelines. 

Information was obtained from SPAs^ through a 
telephone survey and a review of their monitoring 
and evaluation materials. Based on this survey, 20 
SPAs were then examined in greater detail, either 
through a visit to the state or a review otthe instru- 
ments and procedures used in their monitoring. The 
visits included meetings with Regional Planning Unit 
(RPU) personnel and subgrantees.* LEAA guide- 
lines and requirements that affect the management 
and monitoring activities were also reviewed* 

The major tasks facing a monitoring system man- 
ager, it ivas found, are: 



• to establish agreement with the SPA mana 
ment on what monitoring information is n 

• to e$tah|Iish agreement withjthe subgrtalicr on 
what will be monitored; 

• to develop procedures to produce the type and 
quality of information required; and 

• to assure the utilization of the monitoring infor- 
mation produced. 

Chapter II discusses LEAA*s monitoring require- 
ments for the SPAs, Chapter III discusses the four 
tasks mentioned above and the need to complete 
them. Chapters IV through VII offer guidance on 
how SPAs can perform these tasks and develop the 
monitoring capability required by LEAA* Detailed 
examples of current SPA instruments and procedures 
are presented in the Appendix, 

While the approaches to monitoring and the devel- 
opment of monitoring systems are discussed here in 
terras of the LEAA program, they are applicable to 
other organizations operating decentratizcd grant 
programs* 

^ A Regional Plsnninf Uttit (RPU) i$ a roprcKnUtlvt 
body of a unit or combination of units of local govcmment 
which mht$. the SPA In its comprehtniive planning by 
providing Information on local criminal Jutticc lyitcm 
needs, and to support thii activity, receives federal funds 
from the SPA. An RPU may also be given additional 
respontibilitki» such at involvemeat In the development or 
review of local subgrant application!* management of tub* 
granU and project monitorlng> ' . . 

A subgrantee it a recipient of Federal funds from the 
SPA (the grantee of LEAA) to carry out a criminal jiuUce 
pro)ctt. It can be a unit of local or State government or a 
non^governmental group. <^ 
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II. MONITORING REQUIRgMENTS 



This chapter discusses LBAA's requirements for 
monitoring by SPAs, defines the nature and scope 
of the required monitoring, efnd describes its rela* 
tionship to management and other types of evalua- 
tion. 



A. LEAA Rtquir^mtnts 

The evaluation guidelihes for SPAs proposed by 
the LEAA Evaluation Policy Task Force- and 
adopted ' by LEAA* contain three requirements 
directly aififeciing monitoring; 

* • *'The SPA^hall insure that the subgraiu appli- 
cation and the subgrant process provide the 
prerequisites for an internal assessment of each 
project by the subgrantee as well as more inten- 
sive monitoring and evaluation activities as 
determined by the SPA/' 

• "The SPA shall monitor the implementation, 
^ operation and results of the projects it sup- 
ports/' 

• "Such monitoring must compare actual activi- 
ties carried out and results achieved with the 
activities and results originally specified in the 
lubgrant application/' 

As an indication of the activities that can be used 
to carry out these requirements, the guidelines state 
that the monitoring may include: 

• "Periodic site visits and interviews with project 
staff/* 

. • "An examination of objective and subjective 
results of the project/' 

• "An assessment of the progress airid the prob- 
lems of the project to date/' 

• 7h9 Report of the LEAA Evaluation Foiicv Task Force, 
VS. Department of Justke. Uw Enforcement Assistance 
AdtnkiirtfatioD. VS. Government Pfintint Office. March 
I, 1974* . 

*LEAA Guideline Manual, M4100 JC (f Proposed Change 
1). VS. Department of Justice, Law Enforcement Anil* 
niniitrition. July 15. 1974, Ptrigftpb 25. 




• "'Effective reporting procedures documenting 

^ project performance/' 

The purpose of such monitoring, according to the 
guidelines, is "to ensure that SPAs generate adequate 
information to carry out their management respon^ 
sibilities" and "to have performance information 
utilized in planning and decision making in order 
to assist program managers to achieve established 
goals/' 

The LEAA guidelines require a radical departure 
from what many SPAs have considered to be moni- 
toring. **Monitorbg/' to them, has meant simply 
information gathering, such as describing items 
bought with troject funds; comparison of planned 
and actual results has been considered **evaluatJOn/* 

But under the *new LEAA guidelmes, monitoring 
involves describing planned project results and com- 
paring these with actual achievements; evaluation is 
viewed as a more intensive analysis, utilizing more 
accurate or conclusive information to verify that 
changes or achievements are, in fact, attributable to 
project activities. Intensive evaluation typically in- 
volves such techniques as experimental designs and 
control groups* 

For example, a sc.hool counseling project might 
be designed to reduce the misdemeanor arrest rate 
among participants by 50 percent. By monitoring 
actual arrests, an SPA could detect whether the 
expected reduction in arrest rates did occur* If the 
rate did decrease as expected, officials muy be willing 
to presume that the project caused the reduction and 
judge the project a success* If the me did not 
decrease, remedial actions or project modification! 
may be initiated. Howe>^r, to verify that a change ^ 
In arrest rates is attributable to the project, the SPA 
may have to design an evaluation involving piurtici- 
pants and non*partieipants in the project and com- 
pare changes in arrest rates for the two groups. 

Developing the required type of monitpring is 
comphcated by tlie fact that LEAA's enabling legii- 
lation gives SPAs wide latitude in setting objectives 
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imd fundirig projects.* It pcnnits "any activity per, 
taining to crime inrevention, control, or reduction 
or the enforcement of the criminal law." In general, 
the SPAs have not limited this discretion within their 
own.state-and all SPAs operate a very diversified 
program. The resultmg diversity of SPA programs 
from state to state and within states makcs^ 
ficult to devise standard guideUnes for nieastiraig 
performance or incorporating information into deci- 
sion-makmg procedures. Therefore, the development 
of monitoring systems in SPAs requires an under- 
standing of the relationship of monitoring to man- 
agement and to project descriptions and evaluation. 



B. Rdatianship of Monitoring to / 
Manogomont 

The management of an SPA can be characterized 
by its objective* and by the actions it takes to achieve 
them. A Qrpical objective might be **to reduce 
specific types of crime by drug addicts by a certam 
percent during a particular time frame." Typical 
actions might include funding projects that hold 
promise of achieving such an objective, providing 
technical assistance to -those that need additional 
help and canceling those projects th**t fail. 

Monitormg provides one type of information upon 
which management actions can be based. Specifi- 
cally, monitoring* provides current information to 
management on the implementation, operation and 
jpmiediatc output of a project while it is in progress. 
When any of these is judged inadequate, manage- 
ment can take corrective action to increase the 
chances tiiat tjie project will satisfy the SPA's objec- 
tives and goals. In. the example above, monitoring 
should detect when the anticipated drop in drug- 
related crhnes fails to occur or when actions designed 
to cause it are not occurr'mg, and indicate when tech- 
nical assistance is warranted. 9 
The resulting action — ^such as modification of on- 
going projects, continued funding, cancellation, or 
planning for future projects — is left to the SPA's 
discretion. But the guidelines require that the SPA 
Comprehensive Plans submitted to LEAA describe 
"how and when monitoring information will be used 
to modify the operations of projects and affect the 
planning and funding decisions." Each SPA will, 

-•US. CongrcM. Crime Control Act of 1973, Title I, 
Public Law 93-83. H.R. 8152, August 6. 1973. "Part G— 
Definitioiu, ptragmph (a)." 



therefore, have to specify the relationship between 
monitoring and management. Guidance on how the 
SPA can accomplish this task is pven in Chapter IV. 



C. Relotibnship of Monitoring to Profict 
DtKription and Evaluation 

The LEAA guidelines state that the basis for 
monitoring is the project description pven in a grant 
application and that an cvriuation design is an 
hnplicit part ot the project description. Such a. 
project dwcription should outline a sequence of 
activities to be pursued and a set of expected result». 
This simplified diagram depicts a drug treatment 
project: i * 
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The series of events (boxes), and Uic assumptions 
that one will resuU in tiie next (ai?ows) represent 
the logic of the program! Once the events are speci- 
fied and levels of achievement projected, the events 
can, be monitored to determme whether they actually 
occur. Such a project description thus determmes 
what Is to ^-monitored and provides standards for 
measuring achievements. ^ 

Intensive evaluation, on the other hand, can be 
used to determine whether the logic itself is correct 
—that is, wheUier one event can be attributed to 
ainother. For Example, „ an ^ evaluation may ^^t 
whether the above drug treatment project cmwedf 
a reduction in crime, or it may test whether the 
project caused a reduction in drug addiction among 
those treated.* 

' ^' 

«Thc first example would normally be called an "impact" 
evaluation, in that it tests the impact of the project on the 
social environment (in this area, the crime rate). The 
secoMd example, which looks only at the direct effects of 
project activities {It., changes in project participanU) 
would be called an **effcctivcncsi" evaluation. 
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Momtoring i$ not necessarily distinguished from 
intensive walut^on by the events on which informa- 
tion is collected A project can be monitored in terms 
of resources expended, activities implemented, out- 
puts produced, project objectives achieved, and SPA 
goals achieved. The LEAA guideUnes require that 
SPAi monitor at least project activiti^ and those 
events that result from the activities. In the above 
example, this would require that at a minimum 
the project output—change m dependence of project 
participanU on drugah-bc monijtored. Guidance on 



how the monitormg system manager can determine 
exacdy which events should be monitored is given 
in Chapter, V. . , 

In summary, LEAA is not only requiring SPAs 
to monitor all projects they fund, but to monitor 
them relative to the activities and results which the 
subgrantee proposes to achieve?. This will require - 
many SPAs to broaden their current information 
collection activities into true performance monitor-^ 
ing. ^ 
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III. MAJOR PROBLEMS CONFRONTING A - 
MONITORING SYSTEM MANAGER 



A variety of problems are being encountered by 
those who design, operate and use monitoring sys- 
tems. While some arc procedural issues unique to 
a specific SPA or RPU, most are symptoms of fun- 
damental policy, organization and technical ques- 
tions which need to be addressed before useful 
monitoring systems can be developed. This chapter 
identifies those questions and the current situation 
of SPAs relative to developing a monitoring system. 
In many cases, the monitoring system manager does 
not have the authority to resolve these questions; yet, 
he must deal with them. Subsequent chapters discuss 
how he can proceed in this ambiguous environment 
to develop a useful jnonitoring system. 



A. Four Major Tasks Facing the SPA 
Monitoring System Manager 

The four major tasks that face an SPA staff 
attempting to develop or modify a monitoring sys- 
tem, as previously noted, are: 

• to establish agreement with the SPA manage- 
ment on what monitoring information is needed; 

• to establish agreement with the subgrantee on 
what will be monitored; 

• to develop procedures to produce the type and 
quality of information required; and 

• to assure utilization of the monitoring informa- 
tion produced. 

Monitoring system managers have little guidance 
or precedent for carrying out these tasks and have 
difficulty in both defining and executing them. Ac- 
complishing each of thi^se four tasks will require a 
significant investment of a monitoring system man- 
ager's time and resources. 



B. Current Situation of the SPAs Relative 
to Developing a Monitoring System 

Typically, one person or organizational unit in the 
SPA is given overall responsibility for designing and 
managing a monitoring system. The monitoring sys- 
tem manager often is constrained by three condi-' 
tions arising from the nature of the LEXA grant 
program: 

• SPA program and management policies are 
often ambiguous, making it unclear what is 
to be monitored and why. 

• The monitoring system must often be relate^! 
or linked to other SPA functions (e::g., plan- 
ning) that are the responsibility of other orga- 
nizational unili^This raises issues of communi--. 
cation, information flow and, often, SPA 
organization. 

• Technical problems of measurement and in- 
strumentation are compounded by the fact that 
many SPA-funded projects are unique. 

Though these conditions ultimately determine the 
success ofihe monitoring system, the manager usual- 
ly has little control over them. Resolution of these 
conditions depends directly on successful perfor- 
mance of the four tasks identified above. 

Task L Establish Agreement With the 

^ User on the Information Required 

The first task is to determine who will use the 
monitoring information and obtain agreement with 
them on their information requirements. Success in 
this task is critical because there is little present, 
agreement, opinion or guidance within the SPAs on 
what monitoring information should be produced 
and for what purposes. 

Experience indicates that the SPA management 
itself often cannot articulate information require- 
ments. The monitoring system manager therefore^ 
must develop a strategy for interacting with SPA 
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management and perhaps influencing management 
procedures. 

Task 2. Establish Agreement With the Subgrantee 
on What Will be Monitored 

Planned project activities and results frequently 
arc not described in sufficient detail to permit an 
objective determination on the extent to which they 
arfe being achieved. Mdnitors often have a general 
opinion that all is not gomg well, but lack criteria 
agreed to by the subgrantee to support such opinions. 

Agreement with the subg?rantee on what^'is to be 
monitored is criUcal since the SPA docs not exercise 
durect management control over the project. If the 
SPA management intends to hold subgrantees ac- 
countable for specific activities and results, they must 
specify them beforehand. Otherwise the SPA monitor 
cannot decide what information should be collected. 
Task 3. Establish the Information Flow 

Due to the decentralized nature of the LEAA 
block grant program, obtaining and processing moni- 
toring information often is complicated by lack of 
control over primary sources of data and the wide 
range of information required for a diverse set of 
projects. These conditions have forced SPAs to 
develop a yariety of data collection instruments and 
make it difficult to manage the information flow 
process.' 



^ In many SPAs, monitoring is equated with this 
•information flow process and, in fact, data collection-- 
and processing account for the bulk of the expense* 
and most of the problems in operating existing 
monitoring systen^s. The frequency with which SPAs 
alter their data collection pjfocedures is aa indication 
of the difficult mature of this task. 

Task 4. Assure Use of the Monitoring Information 
The final task is to see that mdnitoring informa- 
tion is used by those who need it. Often, much of 
the monitoring data that is collected is not considered 
by management. The use of monitoring information 
is inhibited by the fact that management is not 
accustomed to having reliable data on projects, and 
many SPAs typically have a high turnover m staff 
and management poHcies. For these reasons, it is 
essential to establish monitoring as an integral and 
continuing parr of the management system. Several 
monitoring system managers have found it necessary 
to develop special procedures to motivate or force 
management to utilize monitoring information. 

The need to perform these four tasks results from 
the diversified nature of the SPA programs and the 
type, of Monitoring required by LEAA. While the 
tasl^s are discussed separately here, the outcome of 
one affects what is involved in the .others and the 
tasks heed to be performed in an integrated and 
complementary manner. 



IV. TASK T-ESTABIISHING AGREEMENT ON 
INFORMATION REQUIRED BY THE USER 



Existing SPA statements on the purpose or use of 
monitoring arc typically imprecise and offer little 
guidance to a manager developing a monitoring sys- 
tem. This chapter discusses how to determine what 
. monitoring information, if any, is required by the 
SPA, and hbw the monitoring system manager can 
develop a consensus in SPA management on what 
monitoring information should W jproduced* 



A. How SPAs Currently Um Th^ir 
Monitoring Systoms 

Experience show^^Aat specification of monitoring 
requirements is a Iqlnjj and difficult process because 
in most SPAsi J 

• management objectives are vague, 

• information requirejgents are not easily artic- 
ulated, organizational structures which could 
act consistently upon monitoring information 
are lacking, 

• management policies and personnel are con- 
stantly changing. 

The type of guidance a monitoring system manager 
will receive when he attempts Task 1 can be inferred 
from current SPA practices; Table 1 Presents the 
seven most common uses of information as indi- 
cated by SPA staffs and documents. They have been 
broken down by objective, primary user, and action 
taken by the SPA. 

(1) The first use— to meet Federal requirements 
for a monitoring system — is now common to all 
SPAs, To meet it, the SPA must simply have a sys- 
tem (forms, procedures, reports, files . . .); Federal 
requirements provide little guidance on content or 
design. 

(2) The second use is to meet infortaation re- 
quirements imposed by such organizations as 
LEAA, thp govemor*s office, state legislature, local 



government or citizen groups. But these i^uests are 
frequently so general that the^SPA cannot determine 
what specific information will satisfy the demandi. 

(3) The third use is to identify technical assil- ' 
tancc needed by a subgrantee. Such assistance varies 
among states and includes: idetttificatiott U operat- 
ing problems in die project, advice on how to opwatc 
projects, advice on liow to deliver specific services, 
etc. 

(4) The fourth use is to provide t documented 
record of a project's operation for such specific 
funding decisions as: cancellation of a project; re- 
funding of a project with SPA monies; and trans- 
ferring funding responsibilities to local or state 
governments. 

(5) The fifth use is to guide the *»ture design^ or 
funding of similar projects. Monitoring information 
seems io )k of greatest value for this piirpose when 
past projects have operated extremely well or ex- 
tremely poorly. 

(6) The sixth use is to help subgrantee managers 
measure their own progress. Many subgrantces do 
not have sufficient experience to set up a monitoring 
system themselves. The SPA can help identify items 
thii should be monitoredj data that should be col- 
lected, and techniques for processing the data to 
produce useful information. 

(7) The final use is to identify projects that are 
underspending their allocated funds. The unused 
funds can then be reallocated or the project can be 
modified to take advantage of the unused portion.* 

All of the above are valid uses of an information 
system. But they do not offer tiie guidance which a 

« It it considered cmbwrtaing to leUim undent moMy 
to the treasury while, at the lamc time, SPAs receive mort 
project propoali than they can fund. Once the SPA jprtnts 
mtnaier hai identified that excess fuadt will be tvailabto, 
then the SPA can determine ways to utlllto thme fundi* In 
teveia! states vUlted, the mo«t Important achievement at- 
tributed to their moflitoring lyttem was the IdeotWcatkm 
of projects that were operating in luchb a manner that not 
all fundi committed to the grant would be expended. 
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TABLE It Sr A IJSES or MONITOIUNG SYSTEMS 



5FA Obftctive for the ^ 
Monitoring Syftefn 


Primary Vgtr of the 
Monitoring information 


^ Type of SPA Action Taken 
on the Monitoring Findings 


*• Au iiicvi icii!(!r«i rcquireinenu tost tnc SPA nsvc a 
monitonng sytt^m^^ 


SPA Director 


None 


2. To meet rcquircmcnu^or rcquoili for infonnttion oo 
- tubgrtntcc activities placed on the SPA by ouUide 
orfanuations (ej., LEAA, State Legtilature). 


SPA Commission or 

Council 
SPA Director 
SUte and Local UniU of 

Government Staff 


Dissemination of information to 
organizations and group re- 
questing/requiring it. Informa- 
tion is used to demonstrate that 
the SPA knows what is hajijpcn* 






ing in its projects. 


3, To allow a project every opportunity to achieve lU 
stated objectives. 


SPA.Stair 


Providing technical assistance to 
projects judged to be in trouble 
or performing poorly. 


4* 10 Shift funds away from poor performance projecU 
or assure continued funding to projecU with the poten- 
tial for high performance* 


SPA Commission 
SPA Director 
dCate ana Local Units of 
Government 


Reprogramming of SPA grant 
funds among ongoing and pro- 
posed projects; 


5« To design future projects. 


SPA Planning Staff 
Subgrantee 


Experience of ongoing and com* 
pleted projects used to shape the 
design of similar new project;s* 


6. To provide projecU the management tools necessary to 
* achieve their stated objectives. 


Subgrantee 


None 


7* To mmimizc the amount of funds returned to LEAA. 


SPA Commission 
SPA Director 


Reprogramming allocat^ funds 
that otherwise would not be ex^ 
pended before the authority to 
obligate them expires. ^ 



monitoring system manager needs to determine 
exactly what information should be produced. Pres- 
ent systems typically arc not linked to any man- 
agement program; they operate as separate staff 
activities unconnected to user requirements. But 
more importantly, for the last six objectives listwl, 
it is generally impossible to tell whether existing 
monitoring systems are performing well. One reason 
if that many state systems are too new for such 
judgments* But the primary problem is that no^e 
of the last six objectives is stated in a way that per- 
mits measurement of progress. SPAs have not defined 
the situation the monitoring system is addressing in 
measurable terms or agreed upon measures of per- 
formance for the monitoring system. Almost no SPA 
has a record of specific actions resulting from project 
monitoring or of how monitoring information was 
actually used. When asked how a system is useful, 
SPA staff typically respoiid with isolated anecdotes 
about how a particularly bad project was uncovered 
and modified* 

« 

The inadequacies of present "monitoring*'~laclc 
of integration mto the management process, absence 
of detailed information requirements, constant 

8 



changes in personnel and policies — are so great that 
most existing monitoring practices cannot offer the 
SPA monitormg system manager the guidance he 
needs to develop a new system. To get it he must 
^act to (1) develop some agreement on SPA objec- 
tives, actions and criteria, and (2) specify these 
factors in suflScient detail to permit design and 
implementation of the monitoring system. How he 
can do this is discussed next. 



B. How tht Mortitoring Systam Manafler 
Con Corry Out task 1 

As just noted, the system manager can expect to 
begin with little or no specification of what moni* 
toring information is required. Yet, to provide direc- 
tion to his efforts, he must find a way to make such 
a determination. 

1. Tii« Itoitt fhft Monitoring Systom Manogtr 
Con Ploy 

The monitoring system manager is in a diflacult 
situation: his job is to design and supply information 
to a management structure which may not know what 
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information it wants or how it would act upon par- 
ticular types of information. Facing this situation, 
the monitoring system manager can adopt certain 
strategics to deal with the SPA management uncer- 
tainty* 

(1) The monitoring system manager can work 
with SPA management to lielp management think 
through its objectives and priorities and agree to the 
.type of informatio^i tiiey require and its intended 
use: In effect, he would be coordinating tiic ^design 
of tiic SPA'S management program. It might inyolve 
reorganization of the SPA as well as specification of 
the monitoring information. 

(2) The monitormg system manager can try to 
^ guess what monitoring information will have tiie 

greatest impact on tiie SPA decision process. He 
can tiien develop that information and aggressively 
disseminate it to users* To do this, he must have 
authority to implement tiie type of data collection 
system he selects. He must also be a good judge of 
die SPA'S management problems, interests and capa- 
bilities. He needs to package results for manage- 
merit in such a way tiiat the action implications are 
obvious. 

A (3) The monitoring system manager can appoint 
nlimself the principal user of the monitoring infor- 
mation and pay littie attention to the rest of the 
management structure. For example, he might use 
his staff to provide assistance to projects in trouble. 
He could design his monitoring system with tiiat as 
its principal use. In these cases, he must have the 
authority and resources to implement the data col- 
lection system and action program he selects. 

The first strategy is ideal since it involves exami- 
nation of tiie whole SPA effort and development 
jof a **rational," well-defined management program, 
with monitoring as one part of a total system. Being 
ideal, it is also the most difficult. All significant per- 
sonnel must be consulted, and complex policy ques- 
tions must be raised and resolved. Such efforts are 
time-consuming and hard to focus because of the 
number of personnel and issues involved. 

The second strategy is^ one usually followed by 
evaluators and information system designers. Frus- 
trated by a lack of clear guidance from users, they 
guess what information is best. But, experience witii 
Federal programs shows that diey often guess wrong. 

The third strategy appears to be a reasonable 
compromise. The monitoring system manager con- 
siders himself ^e user and carefully specifies what 
information he will need and how he will act upon 



It. Systems il^^hich the monitor provides technical 
assistance to projects come close to adopting this 
strategy. 

Whichever strategy tiie system manager adopts, he 
will have to adjust his tactics to certain organiza- 
tional realities. First, his success or failure will often 
depend on tiic behavior of organizational units out- 
side of his control. It tiierefore may be wise to docu- 
ment their behavior in order to account later for the 
performance of tiie system. Second, monitoring will 
be e^sy to write off or ignore if it does not produce 
obvious results quickly (say witiiiti tiirce montiis)* 
This argues for designing a simple system with easily 
met objectives* Finally, tiie changing nature of SPA 
staff and policies creates a need for a continual 
re-examination and monitoring Of SPA information 
requirements. 

2. GuidtlinM to Follow In btoblishlng Moni* 
toring Information toqulromonts 

In all tiu'ee strategies tiie manager needs to 
develop new and specific statements of information 
use as the basis for tiie design, implementation and 
management of a monitoring system. As noted 
earlier, most current statements of use are inadequate. 

The monitoring system manager can follow thr^e 
guidelines to determine when use statements are 
adequate: 

• test the acceptability of tiie monitoring product 
to tiie user, ' 

• test the feasibility of obtaining tiie monitoring 
information, and 

• examine the monitoring system design to deter- 
mine if it can be evaluated as part of a man- 
agement support program. 

If tiie monitoring system meets tiiese guidelines, the 
manager has a useful description of information 
requirements. 

a. Testing the acceptability of the monitoring 
system product. The amount of monitoring data given 
to a user can range from site visit interview sum- 
maries to action recommendations. Whatever tiic 
form, its usefulness will depend in part on tiie user*s 
confidence in it and his ability to understand it. 
This means tiiat, during tiie design phase, tiie moni- 
toring system manager will have to work with tiic 
user to test whetiier the monitoring information will 
be acceptable. Oi>e test would be to provide samples 
of information to be produced and have the user 
attempt to act upon it. The monitoring system man- 
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ager needs to challenge users with, various options 
to obtain agreement on what wilt be acceptable 
iftformation. 

b. Testing the feasibility of obtaining the moni- 
toring information. The monitoring system manager 
also must assess the feasibility of obtaining the 
promised monitoring information. For example, if 
the user wants **expert opinion'' the manager must 
determine whether the staflf can provide such opinion. 
In many SPAs, monitoring personnel are assigned to 
projects on the basis of their geographic location 
instead of their technical expertise in relation to the 
project content. Consequently, they may be assess- 
ing all types of projects in terms of their success in 
merely generating certain activities, but often are not 
qualified to assess the likelihood that certain objec- 
tives will be met. On the other hand, the diversity 
of projects may prevent the use of a standardized 
reporting system. 

Testing the evaluability of the monitoring sys- 
tem as a part of a management support program. 
K a monitoring system can be evaluated as part of 
a management program, its manager can be con- 
fident that the system is logically insistent and 
well-defined. For, at a minimum, an evaluation 
design requires that the monitoring system's objec- ^ 
tives and activities be measurable and linked by 
testable hypotheses. If these conditions exist the 
manager has a basis for assessing how well the 
system is being implemented and how successfully 
it is performing. Here we illustrate how an evalua- 
tion design for a monitoring system can be developed 



and utilized to assess the sc^undness of the system*s 
design^ 

Figure 1 gives a simple flow model of a monitor- 
ing system as part of the management support pro- 
gram. As indicated in the figure, the monitoring 
system is used *y the SPA to initiate actions ex- 
pected to increase the chances that stated SPA 
objectives will be met. It is presumed that the SPA 

• has adopted a set of objectives, 

• has developed a set of information requirements 
for determining whether projects are operating 
in a manner that will contribute to achieve- 
ment of the objectives and 

• will take remedial action when the inforjtnation 
indicates that a project is not likely to con- 
tribute to achievement of the objectives. 

The role of the monitoring system is to provide the 
required information. As shown in Figure 1 the role 
consists of collecting and analyzing data from on- 
going projects in order to draw conclusions on the 
status of projects relative to the stated SPA objec- 
tives. The ^'analysis" step produces the information 
required by the^user of the monitoring system as*a 
basis for making decisions and taking actio/Rraimed 
at achievement of the objectives. Each element of 
Figure 1 should be defined in sufiici^t detail to 
provide a model for the implementation of the sys- 
tem and a framework for testing the assumptions 
(indicated in Figure 1 by the horizontal arrows): 

(A) that the user is able to use the information, 

(B) that the actions decided upon do take place, 
and 
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TABLE It THE NORTH CAROLINA. SPA MONITORING 

s^Sl^jSrjStowAMAfiJiGmEtri support program 



Program Compotnent 



Description of Component 
for North Carolina System 



SPA mmnilcnKftt ob>ectivcr which 
monitorinf system is designed to help. 
»chlcve« 



Hive all projccti achieve th« specific 
objectives and goals given in the grtnt 
apphcations. 



Actions taken to increase chances that 
SPA objectives will be met. 



Decisions made in response to monl* 
toring information* 



Conclusions produced by the monitor- 
ing system on the status of projects. 



SPA "Project AnalysU'* act as trouble 
shooters to resolve problems identified 
through the monitoring system thit 
arc expected to impede achievement 
of prpject objectives and goals* 



Evaluation Unit informs appropriate 
Project Analysts when activittes.speci* 
fkd in the grant application and con- 
sidered essential to project success 
either are not occurring as planned 
or are not producing the expected im* 
mediate output The Project Analysts 
are then respon^i^^c for deciding 
whether action is warranted* : ^ 



Standardized forms are used.by RPU 
and SPA sUfTs to collect information 
on a proicct*$ activities and outputs 
given in the grant application. The 
Evaluation Unit compares planned 
activities and outputs with those tc* 
ported to be actually occurring sn or* 
der to identify problems warranting 
action by the SP A>roject Analysts. 



Measures and Instruments 
for Testing Occurrence 



the North Carolina Evaluation Unit 
processes grants to ensure thiit project 
objectives are stated in measurable 
terms and that a methodology tuiiti 
for measuring the a.chievemcnt of 
them, SPA and KPU staff members 
involved' in processing grujits receive 
training Sn what constitutes acceptable 
' objectives. The methodology given in 
the grant application Js used to detcf* 
mine the extent to which project ob* 
iectives a re met. 

Project Analysts provide Evaluation 
Unit feedback on all actions taken by 
completing **liequcst for Evaluation 
Follow*up Fornh** Monitoring system 
can be uKd to determine effect of the 
action. V. 



**Request for Evaluation Eollow^up'* 
form is used by Evaluation Unit to 
bring problems to |he attention of 
Project Analysts and obtain feedback 
on whether action was considered 
appropriate. 



Evaluation Unit tiscs established con^ 
trol procedures to monitor collection 
and analysis of data and whether 
problems are identified and referred 
to the Project Analysts. 



(C) that the actions lead to .aehievcmcnt of stated 
SPA objectives. 
To test these assumptions, the SPA needs measures 
and instruments for collecting data on the lour asso- 
ciated events in Figure it conclusions on the status 
of the projects; the uscr!s decisions; actions taken; 
and the achievement of SPA objectives. 

The testing of the cvaluability of a mohitoring 
system can be illustrated with the system dcvclppcd 
by the Evaluation Unit of the North Carolina SPA. 
Table 2 describes the components of the North 
Carolina monitoring system and identifies measures 
and instruments which could be used to evaluate 
the operation and effects of the system. . * 



In principle, the North Carolina system is cvalu- 
able. Each assumption (A» and C) can be tested 
with the available meajfurcs and instruments^givcn 
in Table 2. An evaluation follow-up form enables 
the monitoring system manager to determine whether 
the user can use the information and whether desired 
actions occur. It may be possible to determine the 
degree to which the SPA objective is ac}iievcd by 
comparing quarterly project progress reports with 
final project results. It would be difficult, however, 
to conclusively test assumption C— that the informa* 
tion and actions lead to achievement of SPA objcc- 
tivei. To do this, one may need to perform cvalua* 
tion analyses such as making 'a comparison of the 

n 



number of projects that achieve SPA stated 'objec- 
tives before and after installation of thempnitoring 
system* If base line data arc npt available, tfe? in*a- 
ager might rely on less conclusive evidence. For 
example, the existence of a large number of projects 
that did require too<Iification$ following monitoring 
but which were judged successful upon their com- 
pletion* 



If the management program is evaluable, as the 
North Carolina one appepu^ to be, and if ril users 
agree ^o it, then the monitoring system manager has 
a framework for developing and managing his moni- 
toring system* If it is not evaluable, then it is ques- 
tionable if he has anything to which he can hold 
the users or use to determine whether the system is 
having the intended effect. 
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V. TASK-2-ESTABLISHING AGREEMENT WITH THE 
SUBGRANTEE ON WHAT WILL BE MONITORED 



This chapter provides the monitoring system man- 
ager with guidance on how to establish agreement 
with the subgrantee on what to monitor. The dis- 
cussion covers what the agreement should include, 
strategies for carrying out the agr-ecment, and tech- 
niques for determining what constitutes ai^ accept- 
iiblc agreement* 

A. Th0 Subgrontee Applicotion-the Vehicle 
for Estoblishing Agreement 

Monitoring, as aefined by LEAA, involves a com- 
parison of actual project achievements with those 
spccitied in the grant application. Therefore, the 
grant application nmst specify the events to be moui* 
tored. Fmtliermore, the LEAA evaluiition guideli^ies 
require that: 

The subgrant application and the subgrant 
approval process provide tlie prerequisites for an 
internal assessment of each project by the sub- 
grantee as well as more intensive monitoring and 
evaluation activities as determined by the SPA. 

These prerequisites shall include: tlie identifica- 
tion of tlie 'problem in measurable terms; welU 
defined objectives of the project stated in mea- 
surable terms; specific indicators and measures to 
be used to assess the results of the project; and 
means of collecting data and information to assess 
the project's performance. 
In most states, the SPA subgrant application 
reqferes project descriptions. Some SPAs also require 
appiiJSints to specify what results are projected and 
what events will be monitored. However, the content 
of subgrant applications vary significantly from state 
to state and only a few meet the LEAA prereq- ^ 
uisitcs for monitoring. Examples of procedures 
used to develop project descriptions for monitoring 
purposes are given in the Appendix and discussed 
next. 



B, How Agreements are l^stobiished 
in Practice 

SP/^ generally use one of three approaches to 
r^ach agreement with subgrantees on the content and 
form of the project description in the grant applica* 
tion. They are distinguished by the relative roles 
played by the SPA and subgrantee: 

(1) The project proposals are initiated and devel- 
oped by the subgrantee. There are few criteria or 
giiideliiies imposed by the SPA on the form and 
content of tlie proposal* 

( 2) The project descriptions are developed joint- 
ly by the SPA staff and the subgrantee. A series of 
negotiations takes place in which criteria are applied 
as to what constitutes an adequate project descrip- 
tion. 

(3) The project descriptions are ^developed pri* 
maiily by SPA staff and placed as requirements on 
the subgrantee. 

Typically, ,the monitoring system manager does 
not have conuol over the grant application process 
and must adapt to the approach in use. If the 
monitoring system manager is in a position to 
negotiate with the subgrantee or specify the project 
descriptions in the^grant application, then he has a 
means of obtaining agreement with the subgrantee 
on what will be monitored. If he is not in such a 
position, then he eitlier can try to negotiate an 
agreement with the subgrantee after the project is 
funded or dejpend on his staff to identify appropriate 
events to monitor as the project proceeds. In several 
states, the monitoring, system manager or evaluation 
director has been able to change the subgiant appli- 
cation or application review process sufticiently to 
obUiin project descriptions which can be used as a 
basis for evaluations and the type of monitoring 
being required by LEAA. 

In many states, the SPAs utilize all three ap*"* 
proaches mentioned above. For example, due to 
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SPA management capabilities and the organization 
of a state*s criminal justice system, one* approach 
may be used on all projects in a given program area, 
s'uch as corrections, and another approach usetj for 
other types of projects. Each approach is discussed 
bdefly in the following sections, * 

1. Little or No Guidance on Content and Form 

While an SPA may have a standard grant applica- 
tion form, many states allow the subgrantce extensive 
fieedoniJin presenting a project description. Once 
submitted, the application is reviewed primarily on 
the basis of funds available for the type of project 
being proposed. The project's activities and expected 
results are not specified and criteria for an acceptable 
statement of these events are not provided. 

Based on llie experiences of states using this 
approach to d^ev^lop grant applications, the pro- 
posals typically do not provide a good basij for 
deteniiiniug wjhether projects operate as intended. 
Often there is no agreement between the SPA moni- 
tor and the si}|3grantee on what the project should 
achieve, and those who receive information- must 
make an arbitrary assessment of project performance 
based on their own experience* and intuition. This 
Situation can lead to inconsistent results. For exam- 
ple, consider the following project observed in one 
SPA: 

A neighborhood house was to be purchased, 
staffed \\/hh counselors and furnished to accom- 
modate juvenile offenders* The stated objective 
was to provide a **viable alternative" to incar- 
ceration in the *'juvenile hall." The project 
description failed, however, to Include any objec- 
tive criteria as to what constitutes a "viable alter- 
native/' The first several monitoring reports relied 
merely on the criteria that the new neighborhood 
house be in existence and functioning; once the 
^ house was purchased, furnished and in tisc^ the 
monitor concluded the objective had been 
aciiieved. Subsequently, the project was assigned a 
new monitor who used different criteria: how well 
the juveniles were progressing through the treat- 
ment process. He considered such factors as length, 
of stay, runaway rates and changes in juvenile 
behavior. On the basis of these criteria, he con- 
sidered the project unsuccessful and recommended 
that treatment be altered. 

The point is not whether one or the other mohi- 
toring assessment was correct, but tliat succtjss or 
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failure of the project hinged on a subjective judg- 
ment of what constituted success since objective 
pre-stated criteria were lacking. 

As the example demonstrates, even when the grant 
applipation does riot contain a detailed project 
description/ performance data can be collected and 
management action taken. Such activities, however, 
would not* meet the proposcfd LEAA guideline re- 
quirement; often they accomplish little more than 
maintaining contact with a project, checking to sec 
whether it is functioning and expending funds as 
described in the grant application, and they do not 
occur t^onsistently. 

2. Content and Form Ne^otioted on the Basts 
of Established Criteria 

Several states have recently revised then: grant 
application form to provide detailed instructions and 
criteria on events to be specified. An example of 
such instructions and criteria is provided by thQ 
following excerpt from the North Carolina SPA sub- 
grant application form: 

Identify performance objectives for the project. 
Perfoi^niance objectives indicate major behavior 
(activities) necessary to conduct the project as 
planned. Each performance objective should in- 
corporate, where applicable, specific behavior, 
the method of procedures to be followed, tunc 
specifications and how achievement of the objec- 
tive will be documented. Performance objectives 
should answer the questions (1) Who? (2) What? 
(3) Where? C4) When? (5) How? (6) Under 
what conditions? (7) Td what level of acceptance? 
(8) As documented by what? For example: a 
youth services center staff will teach 150 school 
personnel how to change unacceptable, illegal 
behavior patterns (delinquent behavior) into 
acceptable, legal behavior patterns by using work- 
^ shops to leach proper use of techniques* A pass- 
ing score of 70 percent must be made by personnel 
before receiving credit, for the workshop* Docu- 
mentation will include attendance records, test 
scores and certification of course completion* 

But merely providing guidance is not sufficient* The 
SPA also must be able to enforce the instructions 
and negotiate monitoring prerequisites. In North 
Carolina, RPU personnel have responsibility for 
working with subgrantees to develop applications 
in accordance with instructions developed by the 
SPA. The SPA Evaluatiuii Unit tlien reviews each 
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application to ensure that the instructions have been 
followed and requests supplemental information if 
the project description does not provide an adequate 
basis for monitoring and evaluation. 

However, this approach is difficult to implement 
because many SPAs do not have the quantity or 
quality of staff to negotiate with each subgrantee. 
Furthermore, several states have discovered that it 
is necessary to work with the subgraptce to develop 
instruments which can be used to collect data on 
the event: i.c,, both the monitor and the subgrantce 
must know what data are being used and for what 
purpose if the agreement is to have any meaning* 
For example^ one SPA has found that, for projects 
to augment detective forces of police departm<^nts, 
it is inadequate to specify '*the number of cas^s to 
be resolved" as.a project result to be monitored. The 

* measure is anviguous and open to various inter- 
pretations. To resolve this difiBculty, the SPA used 
the police departments' record keeping procedures 
to specify the possible outcomes of a case and set 
planned levels of change for each outcome* 
Although negotiating subgrant applications with 

^ detailed project descriptions is time consuming, SPA 
evaluators and monitors find that the resulting proj- 

. ects are more suited to the type of monitoring called 
for by LEAAi 

9. SPA Specif icoHon of th« Pro[«€t Mod«| 

J In some instances, the SPA itself determines which 
elements will be monitored or included in an evalu* 
ation. In most such c£^es, events are identified on 
which the subgrantee must provide specific data and 
information. For monitoring and evaluation pur- 
poses this amounts to a specification of the project 
description. 

One example is the Ohio Evaluation Instruments, 
whiclt subgrantees arc to submit quarterly. The in* 
itruments give specific questions to be answered 
and specific measures on which to provide data. The 
instructions iinplicitly dictate a particular type of 
project description for monitoring purposes. This is 
demonstrated by the instructions accompanjring the 
questionnaire for Crime Deterrence Projects: 

Projects to be covered ^by this questionnaire 
include all those which seek to deter the commit* 
tiiig of certain crimes by increasing' the risk or 
threat of apprehension and prosecution to the 
potential .offender as opposed to reducing the 
causes of criminal behavior. Such projects may 



. educate the public in methods of markihg their 
property for easier recovery or protecting their 
persons or their homes with alarm devices. Also 
included would be efforts to intensify patrolling, 
either by sworn police, auxiliaries,,. or citizen 
volunteers, and to facilitate accejs to peace forces 
by citizenry by the use of 911 emergency tele- 
phone lines. 

Although we lack a proven methodology for 
relating these deterrent methods directly t6^ the 
crime rate, the underlying assumption is that if 
the risk of apprehension and prosecution rises, 
crime should go down. Thus this instrument seeks 
to compare the number of crimes before and dur- 
ing the application of certain deterrent measures. 
Since our crime dAection and reporting tech* 
niques are often far from perfect, one possible 
. foHoFfup to projects of this type would be an 
evaluation of the detection-and-reporting appa- 
ratus in the jurisdiction which ran the project,** 

- For these projects, the project model is, in its 
simplest form:^ 



1 Cmttj 





















It is assumed that the projects and data collection 
procedures can be designed in nuch a way as to 
provide information for monitorhig each of the three 
types of events, In the above example from the Ohio 
SPA, measures of crime activity are to be used to 
monitor project results. The subgrantee is to specify 
the.specific '*crime(8) to be deferred'* and set goals 
in terms **of the sum total of the crime(s) made 
known to police during a particular quarter/' The 
subgrantee then uses a standard form to report the 
actual number of crimes that are reported and 
present baseline data on the number of crimes that* 
occurred during previous quirters. 

Specific measures to be used to monitor and evalu* 
ate "the implementation of deterrent activities'* to 
be funded are: 

• Public Education: the approximate number of 
people reached in the community by methods 
used to inform them of methods or techniques 
to deter crime* Methods for public education 
might include, for example, lectures, movies,. 

. mass media spots, pH^^hlets, posters, etc, 

• Intensified Police Patrol: the total number of 
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additional man*hours provided to the target 
commonHy or jarea by the police agency for 
intensifled police patrol 

• Auxiliary PoUce/Citizm Patrol: the total num* 
Mtt of man-hours provided to the target com- 
nunity or area by volunteer personnel such as 

* off-duty police officers or Civilians, trained by 
the police agency for this purpose* 

• Surveillance Equipment: the percentage of the 
target community or area which is covered by 
surveillance equipment used to deter criminals. 

• Protection Biluipment: the j>ercentage of the 
target community or area which i$ covered by 
equipment used ifco protect persons or property 
such as locks, safes, lights, etc. 

• Hot Line/ Alarm Systems: the percentage of 
the target community or area which is covered 
by communication systems primarily used to 
alert local law enfort^eraent officials of possible 
criminal acts with the intent of deterring rather 
than apprehending tlie perpetrptors. Examples 
would include both "91 1'' emergency telephone 
service pnd high crime area alarm systems. 

As with the crime reduction measures, quarterly 
goals arc to be set and actual achievement reported 
by quarter. Sample questionnaires and instructions 
used in the Ohio system and in those of other 
SPAs using a similar approach are given in the 
Appendix. 

The diversity of projects funded by SPAs has 
presented problems for those states attempting to 
use the third approach to developing grant applica- 
tions. Ohio has developed 23 different packages to 
accommodate tliis diversity and still flnds that the 
questionnaires have to be inodiflcd for niany projects. 
The District of Columbia had a system similar to 
that used by Ohio but found tliat it did not produce 
the type of information wanted. It has changed to 
one that resembles the sea*nd of the above men- 
tioned approaches. 

Another problem encountered with standardized 
project descriptions is in obtaining clear and con- 
sistent agreements with subgrantees on exactly what 
is to be reported. For example* specified measures 
are frequently misinterpreted even though defini- 
tions are provided. 

As with the negotiated approach, this third ap- 
proach places heavy requirements on the SPA staff. 
Project designs or instruments specifying the mca* 
suies must be developed* and assistance fjmst be 
given, to subgrantees to ensure tliat they understand 
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the requirements placed bn themT To date, SPAs 
have had little experience with this approach to 
monitoring. Some have specified project descriptions 
or events to be monitored for one or two project 
types, but only a few have recently done so on a 
comprehensive basis* 

Of the three approaches to developing a grant 
application presented here, the last two appear to be 
the most useful for performing the type of monitormg 
required by LEAA. The first approach, although 
currently encountered in many, if not most, SPAs, 
will normally not satisfy the new UpAA require- 
ments for monitoring. 

However, problems are encountered when using 
each of the above three approaches to develop proj- 
ect descriptions that will be a^rceptablc for monitor- 
ing purposes* Furthermore, tliere iure fcw*criteria 
to use in determining when a description is adequate, 

C, Tools/Crittria for Spacifyino th* 
Atctptability of a Pfojoct DtKri Jon 

A project description identifies a series of events 
describing what the project is to accomplish and 
how. These descriptions can vary in terms of the 
events specified, the schedule given for the occur- 
rence of evefltii, the measurability of events, and the 
expectations for events. The monitoring system man- 
ager must decide w^hat events to monitor and what, 
type and amount of information to collect. In doing 
so, he should raise two basic questions to assess the 
adequacy of the information requirements derived 
from the project description and the selection of 
events to be monitored: 

• Are tlie niipnitored events necessary for the 
success of the project? 

# Will the information requirements result in a 
specification of measurable events, schedules 
and expectations which will support the dcci» 
sion criteria agreed to by tlie information user? 

1, EvanU Singltd Ou\ for Monitoring 

There aie usually a large number of activities and 
statements justifying SPA*fundcd projects. To decide 
which events to monitor, t^ie manager must go back 
to the "logic** of the project design (i.e., the assump- 
tions linking the expenditure of resources, activitieis 
and results) and identify those events necessary for 
project success. 

For the SPA monitoring system manager, project 
success is defined in terms of achieving SPX objec- 
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tiyes« iThis may present a problem when SPA ob* 
jectives are different from or broader than project 
objectives* For example, a project objective may be 
to provide an alternative treatment process to exist- 
ing methods' of handling drug addicts that is at least 
as effective as existing methods in terms of reducing 
addiction, while the SPA objective may' be to achieve 
a reduction in the number of crimes attributed to 
drug addicts. Problems arise when project data sys- 
^m^^are developed — ^will the pi^oject support the 
collection of data which do not deal with its primary 
obj'i^ctive? Ideally, the monitoring system maltiager 
would develop a monitoring system to acconmiodate 
both objectives; however, this is not ^Avays poss'^e 
ttid a decision needs to be made on* which objective 
will provide the basis for monitoring. 
" Once the criteria for project success are estab* 
lished, then the manager can determine particular 
activities and results that^need to be achieved if the 
project is to succeed., * 

2. Determining What MeosurM Will b« Used 

Once the events to be monitored have been deter^ 
mined, measures need to be specified. Both C|ualita<- 
tive and quantitative measures can be tised. For a 
dnig treatment projqct, qualitative measures might 
include descriptions of the services provided, their 
sequence of occurrence, and characteristics of the 
participants. Quantitative measures could include 
the number of participants served and the amount 
of time or money expended on ea^h part of the 
treatment process. 

In specifying measures, three factors must be 
agreed on: 

• a definition of the events 

• a technique for measuring occurrence of the 
event, and ' 

• a set of procedures and instruments for obtain- 
ing and reporting the measurements. 

Frequently, items to be measured are defined but 
measurements are never obtained, because, it is not 
feasible to obtain the data required. 

If ther^ are to be objective comparisons of what 
is achieved with what was planned, the planned 



achievement levels must be stated before the project 
b^^s and in terms* of the measures to be used for 

mdhitot^ng. A variety of techniques can be used to 
specify these levels. For example, national standards, 
such a^ those suggested in the Report of the Standards 
and Goa^s Commission,^ are utilized by some SPAs. 
If similar projects have been operated successfully 
in the* past, their performance can be taken as a 
standard. However, in most cases the planned levels 
'of achievements are negotiated by a subgrantee and 
t^e SPA aiid are based on estimates of what appears 
to be reasonable. Normally subgrantefes are reluctant 
to specify expected results in* measurable terms; in 
such cases, it is necessary to force them to at least 
make tentative projections with an understanding 
that they can be modified if they tiHrn out to be 
unreasonable. In several SPAs, projects are allowed 
to operate for a few months, then are analyzed to 
set realistic levels of achievement. 

Specifying when these levels will be reached is 
also part otthis task. The activities and results ex- 
pected to occur in each monitoring period should 
be specified. Techniques for extracting schedules 
and expected levels of achievement are given in the 
Appendix. The above three criteria for specifying 
measures can be applied to determine If a proposed 
schedule and set of expectations are adequately iden- 
tified for monitoring purposes. The ultimate criteria 
for the acceptability of the measures is their relevance 
for the user of the monitoring system and the con- 
fidence that the users place in the measures. 

Once the monitoring system manager establishes^ 
the events to be measured and the measures to be 
used, he can develop data collection \and analysis 
procedures and techniques to produce the informa- 
tion required. Due to the diversity of projects funded 
by SPAs it is not feasible or appropriate to specify 
a comprehensive set of measures in this document. 
However, to better demonstrate what is being called 
for, examples of monitoring measures used in several 
states are given in the Appendix* 

7 Sc,e Reports of the National Advisory Commission on 
Crimintl Justice Standards and Goals: A National Strategy 
to Reduce Crime; Criminal Justice System; Police; Courts; 
Corrections; Community Crimf ^Prevention; and Proceedings 
of the National Conference on Criminal Justice, 1973^ U.S* 
Govcramcnt Printing Office, Washington, D.C. 



Vk. TASK 3-ESTABLISHiNG THE INFORMATION FLOW 



Once agreement has been reached with the user 
and the subgrantee on what information is required,, 
" >^e^px0iJffofiiJf. system manager is faced with the 
procedural and lechnical problem of producing it. 
Activities associated i^ith information flow normally 
account for modit of the expense and problems en- 
countered in monitoring. This chapter provides 
guidance in establishing an appropriate information 
flow by identifying what it shbuld^ include and how 
it can be organized. 

\ A. Informotion Flow— What it InvolvM 

The information flow involves four component 
activities: 

• establishing data sources, 

• collection and transmission of data, 

• analysis of data, and ^ 
m dissemination of analysis to users. 

The task facing the monitoring system designer is to 
close the communication gap between die project 
and[ the user in an efficient and timely manner. He 
must develop information collection and processing 
techniques and assign personnel responsibility for 
each task. 

1. Establbhino boto SpurcM 

Data on flaonitored events can come from a 
variety of sources. Four sources most useful for 
criminal justice monitoring are: - 

a. Project^maintained records. Data produced and 
maintained by the project are normally the most 
accessible source of information. In some cases, the 
^ monitor can use information recorded as part of a 
project^s operation (such as in a "counseling project 
in which progress from one step to the next requires 
passing a test). When the subgrantee docs not record 
desired information or maintain it in a suitable form, 
special record keeping arrangements have to be 
instituted and, if necessary, included as a require- 
ment of the grant. 



b. Records maint(uHed by existing agencies. Many 
organizations that act as subgrantees routinely main- 
tain records containing information useful for moni- 
toring. (For example, a youth services bureau may 
keep such records a^ follow-up reports on the youths 
served.) If agencies other than the subgrantee are 
involved in the project, their records can> in many 
cases, be used for monitoring. For example, if ofi^ 
objective of a youth center were to reduce the tru- 
ancy rate, school records would be a likely source of 
monitoring information. 

c. Persons famUiar with the project. Persons fam- 
iliar with the project appear to be the most frequently 
used source of information in current monitoring. 
Project staff members are frequently asked to assess 
progress. Or participants in a project or persons in 
a position to observe progress, are often asked to 
comment on project perfbrmance (such as asking 
patrolmen their opinion of the services provided by a 
project's police legal advisor). For some projects, 
community reactions are obtained for monitoring 
purposes (such as asking neighbors of a community- 
based corrections facility how well they think the 
facility is operated and what benefits are derived by 
the neighborhood). 

d. Special surveys. Formal surveys of a given 
population usually are part of a larger evaluation 
effort. For example, ui.a project aimed at changing 
community opiniohs of the police, periodic surveys 
are often conducted which can be used both to 
evaluate and monitor the results of the project. Such 
surveys, also can be made a routine part of project 
operations.* 

pifferent projects lend themselves to different data 
sources, and questions of budget and manpower 
often dictate which sources can be utilized. In gen- 
eral, the first two categories— records maintained by 
the project or an established agency — are easiest to 
document and most objective. But if appropriate 
records are not ordinarily maintained, such record 
keeping might represent an extra cost for either the 
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subgrantee or the SPA whii^h could not be justified 
or covered by available funds. Where project direc- 
tors are inexperienced in record keeping, some SPAs 
provide record keeping form^i^^ichelp project direc- 
tors develop them. For exai^le, the North Carolina 
SPA has developed personnel time and attendance 
forms for use by projects in which the expenditure 
of personnel time is an activity to be monitored. 
Several states also make it a practice to work with 
selected subgrantecs to develop detailed data collec- 
tion instruments for use by the subgrantecs in operat- 
ing their projects and producing information for 
SPA monitoring purposes* This approach has pro- 
duced comprehensive and useful information for 
SPAs that have used it. However, developing such 
instruments consumes considerable staff time, and 
many SPAs have been able to use this approach on 
at most a few projects per year. 

In summary, the use of record systems is most 
practical when the project is operated by an estab- 
lished criminal justice agency— sifch as police de- 
partments, correctional agencies or the courts. 
Projects that operate independent of such agencies 
usually must either develop special record keeping 
systems or depend on subjective assessments of 
persons familiar with the project. 

* 2. ColUction and Transmittal of th« Informa** 

tion 

Information is collected in two basic ways: site 
visits and self-reporting. Site visits are normally 
conducted by SPA or RPU staff members and range 
from brief superficial contacts to in*depth reviews 
lasting several days. The Connecticyt SPA makes 
monthly site visits to all projects, but in many states 
most projects are visited once a year — ^if at all. 
Several states have adopted a sliding scale to deter- 
mine which projects will be visited and how often. 
For most of these cases, the frequency is determined 
by the^^iic of the grant, using a scale such as that 
suggested by the National Conference of State 
Criminal Justice Planning Administrators; ® 

• $10,0OO-$25,O0O— one visit during a 12-month 
period; 

^ • Technique of this type are presciUed in An Introduction 
to Sample Surveys for Government Managers (Carol H. 
Weiss and Harry P. Hatry). The Urban Institute, 1971. 

» **Proposcd Revised Minimum Standards for State Plan- 
ning Agencies." Adopted by the National Conference of 
Stftc Criminal Justice Planning Administrators. July 19, 
1973. 



• $25,000--$ 100,000— two visits during a 12- * 
month period; 

• over $100,000 — four visits 'during each 12- 
month period. _ - 

The most common frequency used (by those RPUs 
and SPAs that attempt to visit all projects regularly) 
is quarterly, with visits timed to follow receipt of a 
project-generaled progress report. 

The use of site visits is limited primarily by staff 
resources and travel funds. Extracting a comprehen- 
sive set of information is time consuming. In seyeral 
SPAs or RPUs which conduct in-depth quarterly 
site visits, one staff member is assigned responsibility 
for at most 20 projects. In those SPAs conducting 
comprehensive quarterly site, visits, the visits typically 
are made by a grants manager or planner who exer- 
cises a variety of management responsibilities for the 
projects. ^ 

Self-generated reports are used by many states 
as a primary means of obtaining information on 
operating projects. Typically these states conduct site 
visits only when a project reports problems or in- 
volves a sizeable subgrant. All SPAs have some 
mechanism for subgrantee generated reports. The 
mechanisms vary from simple statements that a proj- 
ect is operating to extensive descriptions of activities 
and results with supporting statistical data. Similarly, 
the quality of the information varies greatly. SPA 
and RPU personnel repeatedly reported problems in 
ensuring the validity and completeness of §ejf-. 
generated reports. Costs, for instance, frequenifly 
limit the SPA control over these reports to simply 
processing them. 

Both narrative and quantitative formats are used 
for subgrantee reports and site visit reports: Several 
states use both formats and no state appears to 
depend entirely upon quantitative reports. Some 
states, such as OMo, have statistical and narrative 
sections in the same report. Others, such as Connecti- 
cut, collect statistical information directly from proj- 
ect directors and narrative reports from RPU site 
visits. Narrative formats are of three types: 

a. Judgmental reports— vMch rely on the project 
director or ttionitor to determine if a project is pro- 
gressing satisfactorily. These reports supply little 
or no information for independent assessments. Per- 
sons making the report arc typically asked to give 
general impressions on the progress of the project; 
problems encountered; and, sometimes the degree 
to which special conditions to the subgrant are being 
met. Instructions for completing the report are fre- 
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quently minimaL 

b* Semi-structured rep^ts-r^v/hich provide infor- 
mation on specific topicstwi^pt necessarily on 
specified events, Information may tie provided on 
^ , progress towai-d meeting objectives without address- 

ing each objective individually. 
* c* Structured narrative reports— which address 

. specific events, usually the objectives and plannect 
activities .identified in the grant application and 
items that the SPA or subgrantee has selected as 

measures^ of project progress. 

» . ■ " 

^ While a narrative report tttay include some numer- * 
ical data, some reporting prpcedures are designed ^o 
that all or most of thfe colletrted information is quan- 
titative. In these reports^ particular measures to be 
used normally are specified. Examples of this type of 
report are the ^'Evaluation Progress^Report" used by 
the Connecticut SPA and the evaluation question- 
naire used by the Ohio SPA. Samples of both are 

' given in the Appendix, along, with examples of nar- 
rative monitoring reports. 

In genera), the structured reports appear to be the 
most useful for meeting LEAA monitoring require* 
ments. If, as LEAA requkes, the subgrant does 
specify items to be monitored, then the monitoring 
report can be structured to obtain information on 
those items. Such reports then provide the monitoring 
system manager with a basis for verifying thi^t the 
desired information and comparisons are being used 

Jn monitoring the project. 

3^ Analysis of Coll«€t«d Information 

Monitoring essentially involves one type of analysis 
-^a comparison of what was planned with what is 
being achieved. The type of comparison is largely 
determined by the standard for the comparison (the 
"planned" results) and the measures of achievement. 

At present, the use of quantitative measures in 
SPA monitoring reports is quite* limited. In most 
caseSi, available information is descriptive and nar- 
rative — siich as descriptions of project activities, 
services provided by the project, and characteristics 
of the project's target population. In these cases, the 
analysis is qualitative and basically compares descripr 
tions of what had been planned with what has 
occurred* Frequently, the descriptions are not suffi- 
ciently detailed or explicit to support an objective 
comparison, and the analysis reduces to a subjective 
judgment on the reasonableness of project activities 
and achievements. This type of analysis would not 
support the monitoring now being called for by 



LEAA. The monitoring system manager should 
therefore attempt to structure the analysis around 
the comparison of objective descriptions or numeri- 
cal data. 

Sbphisticafted statistical analysis is not called for 
irf most cases. The numerical measures to be used for 
monitoring purposes typically involve, stocks and 
flows' in the criminal justice area, such as number of 
persons treated, number of pieces of equipment 
operating, police response times, or number of per* 
sons receiving a service. In iSome cases, rates will, be \. 
calculated, such as number of cases solved per man- 
hour of investigation or cost per person given a 
particular treatment. Such analysis is mainly a mat- 
ter of straightforward arithmetic involving a com- 
parison of project perfonnance data with planned 
levels of achievement to determine whether the 
planned levels have been reached or exceeded. 

4. Disstnfiination of Results to Ustrs 

, In practice, four dissemination approaches arc 
utilized: (1) circulate a package containing most of 
the collected monitoring information, (2) circulate 
a sunmiary report, (3) transmit portions of*the infor- 
mation and analysis tailored to the needs of different 
users, or (4) keep the information on file for dis- 
.semination on request. In the first three approaches, 
the information can be disseminated either On a 
Routine basis or when a particular need arises. 

In the first approach, the circulated package 
usually consists of the original report used to collect ^ 
the information. It may be accompanied by written 
staff conmients. This approach is common where 
monitoring is decentralized and the RPU staff has 
the primary responsibility %r processing reports. 
This method requires relatively little SPA staff tnne 
and effort, but users complam that they receive too 
much information and cannot extract the information 
they need. 

^ ^ Summary reports are designed t6 inform manage- 
ment that a project has been reviewed and to con^ 
firm either that "all is going well" or that there are 
problems. A short description of the project and 
data on a selected Set of measures frequently arc 
presented for backgrbund purposes i or to support 
the findings. '^This approach' is most appropriate 
where a single office coordinates the flow of moniior- 
ing information and where the reports follow a 
commdn format. 

When distinct information requirements are estab- 
lished by different users, it frequently is better to uSe 
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thetlurd method—to disseminate differrent kinds of 
mfocmation to different users. This approach is used 
in several states where an evaluation unit is respoO's 
tible for analyzing monitoring reports arid forward- 
ing selected i^ults to prespecified users. In one state, 
progfam analysts are notified of problems that are 
detected, evaluators receive selected statistical in- 
formation and administrators receive a summary 
report* 

Whichever Approach is used, repdrts can be pro- 
vided either routinely or when certain conditions 
ejdst. For example, several SPAs and RPUs circulate 
quarterly monitoring reports to their councils or 
boards, whereas other SPAs and RPUs do so only 
when a project is being considered for refunding. 

In several stptes, there is very little dissemination 
of results. Collected information is filed and extracted 
only if problems are detected through some other 
communication channel, such as reports in the press 
of a misuse of funds. 

The method most appropriate for a patrticular SPA 
depends on its management style and organization. 
Most decision makers are interested in summary 
reports, or having information tailored to decisions 
they face, whereas, staff members responsible for 
planning projects or providing technical assistance 
are interested in more comprehensive information. 

B, Orgonizatton of th« Information Flow^ 

f Spa monitoring systems.vary significantly in birga- 
^ nization of the information flow. This section discusses 
three models Covering management organizational 
arrangements that monitoring system managers are 
likely to encounter. In general, thc.manager will have 
to design the information flow to accommodate his 
SPA*s management organization* , « 

The models are essentially distinguished by the 
relitionships among the user of the monitoring sys- 
tem, the analyst and the data collector. 

1* If Doto Collator, Analyst and Major Us#r 
ara Hm Sam« Person 

In the first model, one person or organizational 
unit within the 5iPA or RPU performs most of the 
major monitorifig tasks and reconmiends or takes 
iction in resporise to monitoring information. The 
^ monitor Collects the informatio|i through reports 
submitted by the subgrantee, sitd visits or both. He 
then analyzes the information ta reach a judgment 
on the performance of a project. If problems are 



detected, he initiates actions to resolve them. 

The most common example of this approach is 
where one person — usually referred to as a "grants 
manager" — carries out most SPA management func- 
tions for a set of projects. Typically the grants man- 
agers assist subgrantees in developing subgrant ap- 
plications and planning projects, work within the 
SPA to have the subgrant applications approved, 
oversee operation of the projects and intervene in 
those where performance is judged unacceptable. 
The grants managers use monitoring information in 
planning future projects and providing technical 
assistance and may forward it to other components 
of the SPA though typically little or no action results 
from the latter. 

in this model, management actions (e.g., provid- 
ing technical assistance) are an integral part of the 
monitoring system* The extent to which the grants 
manager can monitor and provide assistance to a 
project depends greatly on his own program exper- 
tise. In many large states and in states with widely 
dispersed populations, most grants management as- 
signments are made on a geographic basis, instead 
of according to program area expertise. Conse- 
quently, those monitors face such a wide array of 
project types that often they do not (and probably 
cannot be expected to) have in-depth project exper- 
tise. Much of the reporting and many recommenda- 
tions that result are restricted to items concerning 
project implementation and management, such as 
ensuring that activities identified in the grant appli- 
cation are initiated or that certain records are kept. 
This does not address project results as required 
by LEAA. 

2. if th« Usar is Organizationally Distinct 
From Dota Colloctor and Analyst 

In the second organizational model, the respon- 
sibility for the collection, analysis and use of 
monitoring information is distributed among dis- 
tinct management units. Typically, collection is the 
responsibility of the subgrantee or the RPU. The 
information and the results of any analysis are then 
forwarded to an SPA unit responsible for further 
analysis and dissemination. 

In Connecticut, for instance, monitoring informa- 
tion is collected through monthly site visits by RPU 
personnel and from formatted reports submitted 
quarterly by the subgrantee as part of the SPA*s 
Continuous Evaluation Data Collection System 
(CEDGS). The subgrantee reports are submitted to 
the RPU, where they are reviewed and compared 
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with inforiaatioti from the site visits. If discreptn*- 
cies are detected, th& differeiYCes are resolved bdbre 
the CElXS report is sent on to the SPA Audit and 
Evaluation Division for further analysis and review* 
On the basis of their analysis, the Audit and 
Evaluation Division prepares a summary report 
which addresses such items as: actual progress rela- 
tive to the planned strategy, anticipated impact, 
strengths of the pr6ject, weaknesses of tl^prm^dt^ 
impact upon SPA programming, and gene^^om- 
ments. If outstanding deficiencies are identified, the * 
evaluators formally note it in the Summary report 
and bring it to the attention of SPA planners, project 
personnel and regional personnel responsible for 
monitoring. In general, the evaluation unit does not 
have responsibility for seeing that remedial actions 
are taken. 

In most states, a monitoring report is reviewed 
by at least two offices, each of which can initiate 
management action — such as technical assistance — 
if problems are detected. For example, in states 
where a monitoring report is prepared or initially 
received by the RPU, a state level oflBce typically 
receives and reviews a copy of the report. In Cali- 
fornia, SPA support teams have, in the past, backed 
up and overseen the grants management at the I^U 
level. Similarly, as has been noted earlier, in North 
Carolina the RPUs prepare monitoring reports based 
on site visits. -These go to the evaluation unit, which 
reviews and analyzes them to identify project prob* 
lems that exist or are developing. In some cases, the 
proMcpis are already identified in the RPU report 
ane appropriate remedial action recommended or 
taken. In other cases, the analyses by the SPA staff 
may identify other potential problems. In any case, 
the evaluation unit identifies project problems and 
refers them to other personnel at the state level for 
management action. 

When monitoring reports are used to flag projects 
that need technical assistance, additional informa- 
tion often must be obtained, in advimce of such 
assistance, by phone calls, special site visits or 
reviews of other monitoring reports. An advantage 
of this approach is that an in-depth review is not 
conducted on all projects; and when a review is 
conducted, it can be performed by someone with 
appropriate expertise and be tailored to the specific 
problems that have been identified through the moni- 
toring system. 

•i . 
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3. If th» U^r Rac^iVM All Doto ond OmM^ 
^Out th9 Anolytis . 

In the third model, users are not directly involved 
in collecting information, but do receive most of it 
in order to conduct analysis for carrying out man* 
agement responsibilities. This model is common in 
states which rely on a progress report from the tub- 
grantee as the primary mechanism for collecting 
monitoring information. The quarterly progress re- 
ports are sent directly to SPA planners and man- 
" agers who use them tor refunding decisions, 
preparing information for LEAA, etc. 

This use of monitoring results is also common in 
systems that attempt to collect data on similar 
projects to create a data base for planning purposes. 
The monitoring reports record previous operational 
experiences with specific projects for use in devel- 
oping a basis for planning. While this use can be 
made on a project by project baiis^ the expressed 
intent in many SPAs is to obtain aggregate informa- 
tion on a program basis. Generally, this is not feati** 
ble with systems using narrative reports, which tend 
to emphasize different items on diflferent projects 
and cannot be aggregated. Monitoring systems using 
qommon data collection formats are for the moct ^ 
part new, and it is too early to tell if they will pro- 
vide use;ful data bases for planning purt>oses* 

The selection of a model for organisung monitor- 
mg information will largely be determined by the 
SPAs managanent organization. If the SPA has 
decentralized management functions, then the first 
model (in which one person or management unit 
collects, analyzes and uses monitoring information 
for a given set of projects) appears to be the most 
appropriate. If the SPA has Centralized management 
at the ^PA level but still has a sizeable RPtj orga* 
nizatlon that participates in the management of 
subgrantees, then the second model is the most 
appropriate. This model is alio appropriate for 
SPAs with a hijghly structured staff and management^ 
divided into distinct offices or |inlts. The third model 
appears to be most useful When the SPA has cen- 
tralized the management of subgrants and does not 
work through the RPU in monitoring funded proj- 
ects. Whichever organizational scheme is adopted, 
the raonitoring system manager needs agreements 
with those who Will perform the various informatipn 
flow tasks on what their responsibilities will be, how 
much time will be allocated to each task and when 
each task will be performed. Then procedures must 
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be established for integrating the tasks and ensuring 
that the desired information Vili be obtained. 



C* Functions to bo Performed in Establishing 
tho Information Flow 

In establishing the information flow, there ore 
two basic questions the monitoring system manager 
may use in determining when the task is completed 
satisfactorily: 

• Have tasks, responsibilities, schedules and man- 
power needed to operate the monitoring system 
been identified for each component of the infor* 
mation flow? 

• Is the information produced of sufiicient quality 
to be useful to identified users? 

1. Identify Requirements to Operate the Moni* 
toring System 

To answer the first question the monitoring sys* 
tern manager must have a work plan for operating 
the system. He must be able to identify tasks, assign 
responsibility for performing them, develop a sched- 
ule and specify the quantity and quality of staff 
that will be required. Table 3 is an example, based 
on the North Carolina monitoring system, of the 
type of work plan needed. 

In general, it is not sufficient to merely identify 
tasks and assign responsibilities. Procedures are 
needed to manage the information flow and enforce 
discipline on the system. Several states provide for 
stringent penalties for subgrantees that fail to report 
information or cooperate with monitors. In a few 
states, funds are automatically frozen until reports 
are received. If the condition persists, the grant is 
considered for cancellation. To establish an infor- 
mation flow, many states have found it useful to seek 
the public backing of the SPA commission, the 
executive director and higher state officials. The 
attorney general in one state recently sent letters 
to all recalcitrant projects emphasizing the impor- 
tance and necessity of submitting the requested 
reports. Nearly all reports were soon received. 

A very simple but necessary feature (lacking in 
most states) is a system to monitor the flow of 
information. Several SPAs, and RPUs do not have or 
have only recently established a system to inform 
monitors of such items as when^a report is due, 
when one is overdue, or when one has been received. 
In a few cases, this has resulted in the accidental 



double visiting of a project by two different monitors 
who then filed contradictory reports. While ttiis did 
provide the SPA with an estimate of the reliability 
of the information system, it demonstrated the.SPA's 
lack of control in such simple matters as scheduling. 
Since this type of record keeping is a feature of the 
Grants Management Information System (GMIS) 
being introduced into several states, this problem 
may be solved in those cases. In the meantime, sev- 
eral SPAs and RPUs have solved the problem by 
using coded index cards or display boards to record 
when specific monitoring tasks or activities should 
occur. Some states have secretaries circulate a week- 
ly status sheet with this information on it 

2. Establish Precedure to Exercise Quality 
Control 

In establishing an information flow, the monitor- 
ing system manager needs to know whether accurate 
information is being produced. For this purpose, 
reports by subgrantees are typically reviewed by 
RPU personnel before they are used for manage- 
ment purposes. Generally the RPU staff are in such 
close touch with projects in their region that they 
will be able to determine reporting accuracy. In 
other states, the RPU staff are so involved in projects 
funded through them that the staff cannot review 
reports objectively. In many states SPA personnel — 
planners, grants managers, monitors or evaluators — 
are in regular contact with the RPU staff. It is be- 
lieved that through these contacts, and in several 
instances through contacts with subgrantees, enough 
is learned about the objectivity of the RPU staff and 
about individual projects to be able to sense whether 
monitoring reports are likely to be inaccurate or 
incomplete. In at least one state — ^Michigari^ — SPA 
staff members regiilarly conduct site visits to a few 
projects in each repon to verify the accuracy of 
reports submitted through the RPUs. 

To reduce dependence on the subjective judgment 
and capability of their staff, several SPAs have 
developed standard monitoring reports and criteria 
for managemeht action if probldms are detected. In 
these cases, the critical factor in designing reporting 
forms and criteria is to ensure the reliable collection 
of information and the consistency of the judgment 
process. As noted in Table 3, the North Carolina 
Evaluation Unit reviews all monitoring reports^ sub- 
mitted l^y RPU personnel and critiques them. This 
feedback to the RPUs on the quality of reports com- 
bined with training sessions for monitors, appears 
to be a useful process for improving the quality of 
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the informtticra provided. However, the frequency ncl) demonstrates the dilBcultiei thtt statet have 
with which SPAs alter the procedures and fonns for had in developing an acceptable information flow . 
monitorinfr (due frequently to turnover of person- profcess* 



TABUS M WORK PLAN fOR NORTH CAROLINA MONITORING SYSTEM 



Component 


Tasks 


Responstbility 


Schedule ^ 


A. Eslabltih Data 
Sources 


L Define bow tucceM or 
progreae of inject will be 
metiured and specify 
what records will be kept. 

2* Project initiates data col- 
tectkm technique. 


1, Subgrantee/regional 
project analyst/SPA 4 
evaluation statf« 

2. Subgrantee. 


h During development of 
prt)ject application, 

2. At the beginning of the 

the project. * 


Collect Data 

■4. 


1, Verification of projects 
for which quarterly 
reports are due. 

2, Site visit to project. 

3, Complete quarterly 
subgrant progress report. 


U SPA evaluation chief/ , 
regional project analyst* 

2. Regional project analyst 
3» Regional project analyst 


L Two weeks before end of 
quarter* 

2. At the end of the quarter^ 

3. Within 15 days after ^ 
of quarter. 


C Analvtii of Data 
and Product 


1* Preliminary analysis in 
written report* 

2* Heview reporU substan- 
tively and qualitatively. 


1« Regional project analyst, 
2. SPA Evaluation Unit. 


1, Within 15 days after end 
of quarter. 

2. For two weeks following 
receipt of imports* 


D« DiMcminatioa to 
UKr 

) 


i. Completion of request for 
evaluation follow*up* 

2» Presentation of status 
report on quantity and 
quality of aU progress 
reports to SPA mgmt. 

3. Writing letter to each 
regional project analyst " 
on quality of reports. 


1. SPA evaluations 
assistants. 

2. SPA evaluations chief. 

3. SPA evaluations chief. 


1. Immediately after review, 

2. After review of all 
reports. 

• 

3. After review of all 
reports. 
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VII. TASK 4~ASSURING THE UTILIZATION OF 
MONITOWNO INFORMATION 



Experience dcmonstrttcs that simplyj)roducing 
cvaluttio0 and monitoring informitiptroften does 
not lead to iu utilization. This chapter discusses 
some of the problems which inhibit utilization and 
offers fuidelifies to the monitoring system manager 
for assuring the utilization of the monitoring system. 



A. How tlw SPAi Can Motivoto Utilization 

An examination of monitoring system utilization 
involves two questions: is^ the inlformation acted 
upon, and do the information and actions contribute 
to achievement of SPA objectives? Previous chap- 
ters noted that most SPAs do not operate their 
monitoring systems in such a way that these ques* 
tuMirCan be answered, and only a few states appear 
/fo have the capability to routinely track utilization* 
North Carolina is one SPA that has attempted to 
build such capability into its monitoring system. 

In North Carolma* the SPA Evaluation Unit 
reviews all monitoring repocts. If problems are 
detected, the reviewer fills out an "Evaluation Fol- 
low-up'* form that summarizes the problems add 
recommends action. This form goes to a program 
analyst responsible for providing technical assistance 
to the project. The program analyst acts on the 
problems, then completes the form by giving the 
name of the project personnel contacted and a 
description of the action taken and returns the form 
to the evaluation staff. The evaluation staff keeps a 
log of transactions to provide a record of the utiliza- 
tion process. These procedures ensure that forms 
are not lost and that the monitor is aware of the 
the actions taken to resolve the problems* Subse- 
quent monitoring can determine if the problems 
were in fact resolve^. 

North Carolina's 'ilystem is built for one type of 
management. It has not operated long enough to say 
whether it is well managed, highly utilized and 



effective. But it does suggest an approach toward 
auuring utilization. Other states ttse more informal 
methods to effect utilization. In some states, the SPA 
or RPU executive director discusses neptive reports 
with members of the SPA or RPU board who luivt 
a direct association with a faltering project. The 
implied threat to present the negative information 
to the full board frequently causes th« board mm- 
bers associated with the project to take action to 
mnedy the situation^ Another techniqoe k to rou- 
tmely disseminate the monitoring results to local 
units of government and i^blic interest groajps, a^ 
tactic found especially useful by some SPAs. 

Most states depend on dissemination techniquii 
to motivate action. In this regard, many SPAs have 
found it effective to periodically provide decision 
makers with concise summaries of monitoring re- 
ports framed so as to call for action on the part of 
the recipient. Yet, even when a monitoring systim 
identifies faltering projects, if the SPA docs not have 
the resources to act or procedures for taking the 
action, then no action will consistently result from 
the information. 

Specific examples of management problems which 
inhibit the use of monitoring efforts are: 

No criteria exist fat responding to the monitoring 
information. For example, how deficient should a 
project be before technical assistance is i^ven or the 
grant Is cancelled? If no criteria exist, then informa- 
tion on the level of performance may be useless* 

Resources do not exist for responding to the 
monitoring information. For example, an SPA that 
does not have the manpower or resources to assist 
a project that is having problems will have little use 
for an information system that exists primarily to 
support a management response to projects that are 
not performing as planned* 

Programs and management objectives and man-* 
agement responsibilities are too diverse and numer^ 
ous That is, more questions are asked than can ever 
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be answered and concentrating oh any one set will 
not result in the extensive utilization of monitoring 
results. For example, an SPA that tends to change 
the criteria for program performance every time a 
funded project falters obviously has no predeter- 
mined objectives. 

These types of problems — agreeing on informa- 
tion required by the user— are dealt with in Task L 
Due to communication problems and the continual 
changes in the managements of SPAs, however, the 
problems do not end there. Procedures need to be 
established to accominodate change and verify the 
continued need for information. Frequently, the 
monitoring system manager and the user have little 
contact even though the SPA may have undergone 
several management changes or information needs 
have changed. Moreover, once some information is 
produced, the users may better understand what their 
requirements are. 

Most monitoring systems also have technical and 
procedural problems limiting their use. For example: 

Information may not be in a form that can 
readily be used by administrators and decision mak^ 
ers. One common complajnt by SPA staff is that 
their boards did not review monitoring and evalua*- 
tion results because Reports are tqo long, too in* 
volved or incomprehensible. 

Information does not arrive in time to act. For 
example, due to data collection and processing pro- 
cedures, monitoring results may lag six months or 
more behind an event with the result that it is too 
late to react to a situation-^such as a need for 
technical assistance. 

Projects are funded without explicit statements 
of planned results. For example, no objective cri- 
teria are given to determine if projects are perform* 
ing as planned. 

These problems are related teethe execution o( 
Tasks 2 and 3 — the development of project plans 
against which progress can be measured and the 
establishment of an information flow. However, the 
only sound test of how well these tasks are carried 
out is to evaluate the utility of the product. "Thus, 
the monitoring system manager should set up proce> 
dures to measure use of the information and the 
degree to which SPA objectives are met. 



B. Guidtiiim for Assuring Utilization 
of a Monitoring Systtm 

In Task 1 three guideline^ for developing a moni- 
toring system were put forward: test the accept- 
ability fit the monitoring system product, test the 
feasibility of obtaining the monitoring system infor- 
mation, and make sure that the monitoring system 
can be evaluated as part of a management support 
program. The guidelines for Task 4 involve imple- 
menting an evaluation of the monitoring system by: 

• setting up procedures and data sources to mea- 
sure progress toward SPA objectives, 

• setting up procedures and data sources to mea- 
sure user action on monitoring information, and 

• being prepared to redesign the system if it is 
not effective. 

The ultimate test of the utility of monitoring 
information is the degree to which it '^contributes to 
achievement of SPA objectives* To carry out this 
test, the monitoring system manager must be able 
to measure SPA objectives, record the use of moni- 
toring information and associate the action taken on 
the basis of such information with achievement or 
non^achievement of SPA objectives. 

To illustrate the above guidelines, consider a case 
where the management strategy is to reallocate proj- 
ect funds to reduce the amount of unused money 
returned to LEAA. The jSPA objective can be mea- 
sured in terms of (1) total funds returned annually 
to LEAA, and (2) the number of projects ending 
with funds available that could have been reallo- 
cated. Both sets of data should be routinely avail* 
able from SPA accounting systems. The impact of 
the monitoring system and mana^ment action can 
be inferred by examining the following type of infor* 
mation (where success and failure is defined by the 
project expendmg all its funds): 

• the amount of money unexpended by projects 
earlier signaled by the monitoring system to 
be successful, i.e., expected to expand all funds, 

• the amount of money unexpended by projects 
signaled as not expected to expend allocated 
funds, and 

• the number of projects signaled as not expend- 
ing funds which had money reallocated. 



ERLC 



36 



If the first meiisure turni out to be too high, the 
quality of the monitoring information may be su$«* 
pected and it inferred that the monitoring system 
is not effectively identifying potential problems* If 
^nd measure is too hi^» management action 
be adequate to resolve identified problems^ 
ition such as given by the third* measure 
(^help determine die adequacy and the validity 
monitoring system and the adequacy 6f the 
management action. All three of the measures lAiould 
be built into the monitoring system to test its utility. 

If SPA objectives are not being met— m when a 
large number of projects are not expending all their 
funds--<4he fault could be: 

(1) the monitoring system is not producing Uie 
required information and must be altered* 




(2) the management action is inappropriate or 
inadequately applied and must be changed^ 
and/or 

(3) the objective is unattainable and should be 
" changed. ^ 

Si 

The monitoring system manager will have to decide 
where the fpult lies based on his knowledge of the 
situation* With this experience in hwid, Tasks 1, 2, 
and 3 wQuld have to be executed again* Based on 
previous experience in the LEAA program and 
other Federal and state programs* the^ monitoring 
system manager sh^ld be prepared to routinely 
modify the monitdring lystem based on evaluadon 
of iU operations if continued utilization is to be 
assured* 
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APPENDIX 

EXAMPLES OF PROCEDURES AND MATERIALS USED IN MONITORING 

CRIMINAL JUSTICE PROGRAMS 



Several SPAs have procedures an^ instruments for 
monitoring and monitoring techniques that could be 
used by other criminid justice agencies. Examples ol 
these procedures and materials were chosen to illus- 
trate methods of: exercising quality control over a 
monitoring system; utilizing quantitative reports in 
monitoring; structuring a project tot monitoring; 
and focusing monitoring activities on selected issues. 
The examples presented' were selected fronj a survey 
of SPA monitormg activities. Their inclusion here 
does not represent a judgment that they are part of 
the best or laost effective monitoring instruments or 
techniques. They were chosen solely as illustrations 
ol how certain monitoring tasks could be accom- ^ 
plisbed. Before providing specific information about 
the systems prc;scnt©d here, a brief description of 
the methodology used i^ the survey to (determine 
current monitoring practices is presented. 

Initial contact was made with 38 of the 55 
SPAs through a, structured telephone interview. Staff 
ittcmbcrs M%td background questions about each 
SPA contacted and about monitoring and evalua- 
tion of LEAA funded projects. The persons con- 
i$citd at the^^SPAs include Directors, Plannesi, 
Evaluators, and Grants Managers. Information on 
fflonitorini^ imd evaluation procedures in other SPAs 
was obtained through informal interviews with Re- 
gional OtSqt staffs and SPA staffs. State Compre- 
hensive Plans and documentation provided by the 
states contacted were reviewed. 

Based on the interviews and available documen** 
«ation mne states were visited in an attempt to collect 
more detailed information about their monitoring 
activities. Much of what was learned about those 
activities has been presented in the document. Eight 
monitoring systems, six state systems and two kPU- 
systems from one state, are presented in summary 
form here. 



Many of the states contacted indicated that thek 
monitoring systems were new or had recently under- 
gone changes or were in the process of being designed 
or changed. Most of the systems described in this 
appendix were implemented or changed within the 
past year. In fact, only ope of them hu not under* 
gone some change, including personnel, or hu not 
been effected by SPA policy changes during the few 
months immediately preceding the publication of 
this document. 



A. AtiExampbof Tools ondTMhniquM 
for Controlling tho Quality ond Um 
of a Monitoring Systom 

The Evaluation Unit of the North Carolina SPA 
has incorporated into the monitoring system certain 
techniques to control the quality of the information 
provided by it. In developing the system, attention 
was ^ven to tiie importance of the subgrant appli* 
cation as the basis of monitoring and the corre- 
sponding importance of the regional project analysU* 
role in developing tiie application. The analysu work 
with subgrantees to develop acceptable grant appli- 
cations, and make quarterly site visits to collect the 
data necessary for preparing monitoring reports. 
Although the bwic forms lised for subgrant applica- 
tions and monitoring reports are similar to those 
used by other states, tile difference is in tiie proce- 
dures developed Jfeytiie Evaluation Unit tiiat allow 
a centralized SPAovaluation staff of four to affect 
die cqntent and quality of the applications and 
reports. 

the first quality control measure is the Evalua- 
tion Unit's review of subgrant applications to ensure i 
tiiat tiie prerequisites for monitoring and evaluation 
have been fully developed according to tiie instruc- 
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tions given in the application. Thi< must include: a 
clear statemtot of the problem, a goal statement with 
quantifiable implementation and performance objec- 
tives, specified prefect methods and procedures, and 
an evaluation design. These instructions for the 
application can be found in Exhibit 1. The program 
Component Review of the Subgrant Application 
Processing Form, Exhibit 2, gives the criteria used 
by the evaluation staff to judge as adequate or 
inadequate tiic sections of the application for which 
they are responsible. Unlike many SPA evaluation 
staffs* the Evaluation Unit shares equal authority 
with the planning and fiscal stiffs to reject the part 
of the subgrant application for which it is respon* 
sible and send it back to the project analyst to be 
rewritten. When an application is rejected, it is sent^ 
with the critique and questions of the SPA staff to 
thc^rcgional project analyst who revises the applica- 
tion accordingly and resubmits it. If assistance is 
needed in redesigning a project for evaluation or 
monitoring purposes, the Evaluation Unit will pro- 
vide technical assistance to the regional analyst and 
the applicant . 

The importance of the subgrant application for 
monitoring can be understood in reading the instruc- 
tions of the Quarterly Subgrant Progress Report 
given in Exhibit 3* For example, item VI instructs 
analyst to 'Vefer to goal statements^ implementation 
objectives and pe^ormance objectives as listed in 
, the subgrant application and outlme progress made 
toward their achievement this quarter/' 



In order to obtain the desired .information and 
degree of specificity in these reports, the Evaluation 
Unit has developed two methods forJnteracting with 
the regional project analysts: formal training ses- 
sions and critiques of monitoring reports. Exhibit 4 
is a sample quarterly report prepared for use in a 
workshop seminar with the analysts, and Exhibit 5 
is the accompanying list of criteria which would be 
applied in a critique of a monitoring report for the 
sample project. The sample criteria are given as 
guidelines to assist the analysts as they prepare their 
reports and to provide them with the basis which 
will be used to judge the project and their report. 
The Evaluation Unit reviews all quarterly reports 
submitted by a project analyst and returns a written 
critique of the reports each quarter with suggested 
improvei^ients* Improved reports and more coopera- 
tion from analysts are cited as evidence of the effec- 
tiveness of the feedback process. 

The quarterly reports are also reviewed tq identify 
problems that warrant further investigation or action 
by the SPA* Exhibit 6 gives the form used by tlie 
Evaluation Unit to bring identified problems to the 
attention of those in the SPA responsible for acting ' 
on them and to receive follow-up information on the 
subsequent action taken* This procedure provides 
the Evaluation Unit with a method of monitoring 
the use of their system in terms of affecting project 
operations; North Carolina Exhibits 1*^6 are pre- 
sented on pa^ef 31 through 44. 
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NORTH CAROLlNA-ExhiWt 1 

IN.STRyCTIONfrFOR PREPARATION OF SUBCRANT APPUCATION - PACE 2 ITEMS 

General Inttnicttons « 

The foJtowini part provhle .he writing ip.ce ind imttuctloni for completing eich section of the nirriti*. Jte ^f'^^^*^**^;^ 
TcompJete 'Sfpage .re printed opposite th.t page,. It « suggested that the complete set of instructions «.d »«fore b^^^^^^ 
to write .nd th.t «ch section's instructions be thoroughly reread before writing the respective KCtlon which they coven If mor. 
space is neceiMty. .dd extr. ptges //2 x 1 1 ) ck.ily noting to which section they belong. 

In prep.ring to write this portion of the .ppllc.tion It m.y be helpful to review the m.terials cont.tned ifl the current State Plan. 
Also review local pUns .nd reports which provide b.ckground information In terms of existing raw enforcement and criminal justice 
conditions which affecHor will be effected by) this-project. 

it is imperative that this part of the .ppUcatlon be clear, complete, and epnclse. Unless the project <« q"«l5|y '^'»^'*^*'' 
na aSo will be impoJsible to complete a review and consider the appllction for funding at an early date The »hou^J 
be presented m a nunner that Is easily understood. Each application is judged on Its conceptual merits and the propoKd^ 
administrative structure for Implementation, not on the "gumsnwuhlp" of 'he applicant. 

The application is designed to permit an orderly presentation of a project The Items Identified are those which provide a basic 
dJ^rlJtlon of the structure and intent of the project. Each Item relates to that which precede. It. If one Item is poorly organized or 
haphazardly presented, those which follow will be affected. 

Applicants are advied that the funds available to the Division for making awards are not extensive. Since the f>lnds are limited in 
amount It Is Impossible to meet every identified need or problem for every unit of government. C.r^jhould be e«relsed In 
Sn of pr^applicattons for subnrisslon. because the approval of a submitted application n^^nate or re trtct 
Tonslderatton of future applications from the same applicant. Awards are made only to city and county governmental units and 
state agenciei 



IS Tht IVoWem In this p.rt the probl.m which this project is Intended to addiiss should be described and defined. Accurate problem. 

definltWrTis esKntial becauK the application can only be reviewed and evaluated according to tlie problem It purports to resolve. 

Further. If the application 4s approved and the project funded, the success of tlie project can only be measured irt tetms oflts 

impact upon thai problem. Be particularly careful that the problem deKrIbed Is specifically related to the activities or effort 
' proposed In Items 17-19 of this application. Describe the problem in a manner which offers the potential to observe and measure it* 

dimensions before^ during and after the project activity. 

Asa minimum this Item should: * 

(1) Describe the nature and scope of the problem addresKd in this application » 

(2) Provide supporting facts and figures which describe the e.^tlstence of this problem and a summary of your analysis of the 
' Implications of this information. 

C3) Describe the underlying causes of the problem. 

(41 Provide a clear description of the impact or effect of tlie problem on btJ»er agencies or grdUps. Discuss their awareness of the 
problem. Ihelf need to solve it. and their relationship to your agency as It relates to this problem. 

(5) Provide a summary mdlcatini legal responsibility andior past achievements. e.\peiience. or recent activities which qualify Ihe 
applicant agency to implement the project, 
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INSTRUCTIONS FOR PREPARATION OF SUBCRANT APPLICATION - PAGE 4 ITEMS 

" * ♦ • 

17. Gotl Slitemefit and Memnble Objectives: This section is vita! to the application. It should clearly and concisely present the 
foil statcrticnt and measurable objectives for the project, in other wordS, this section should describe jirccisely what the project will 
achieve and/or demonstrate. The goal staten:f'?nt and measurable objectives presented in this section should be directly related to 
the statement of the problem (page 3) so that the projeci^ can be eval uated in terms of its ability to resolve the problem identified. 
The methods and procedures and evaluation design items called for in the next two sccti-^ns of the application (pages 5 an^ 6) must 
be deiigtied to achieve and document the achievement of the goal and objectives established in this section. 
As a iranlmum this section should : 

(1 ) Specify a foal statement for the project . The goal statement should ckariy communicate the intended result of the project as of 
the end of the subgrant period. The goal statement identifies, before the project starts, what must happen or be achieved in order 
for the project to be considered a success. The goal statement must be* precise enough so that a person could, on the basis of project 
records and data» determine if the project goathas been achieved. ' * 

* Consider the following example of a broad, pooriy defined goal statement: "To decrease juvenile delinquency and juvenile 
problema". Whftjlocs the writer mean by delinquency? Formal charge brought against juveniles in «atc courts? Truanpy? 
Neighborhood complaints? Drugs? Whom docs the writer consider to be a juvenile? Everyone under 21 years? Persons 13 • 18 
years? What is considered a juvenile problem? Criminal problems? Family problems? Employment problems? 
Now consider the foUowing measuraUe goal statement: Twelve (12) months after project implementation in the community there 
will be at least a 20% reduction in juvenile court referrals among.pcrsons (male and female) under the age of 17. Juvenile court 
referrals for the last six (6) months of the project will be compared with the same data from the same period for the previous year. 
It would be possible, with supporting data, to determine if a project witli this goal statement had succeeded. 
Below is one method for writing a precise goal statement: ^ 

(a) Identify the tcrmirtal (end) behavior or condition which will be accepted as evidence that the project his achieved its goal, 
.(In the example above* reduce juvenile court referrals) I 

(b) Try to further define the desired behavior or condition by describing the important limits or circumstances uridcr which thft 
behavior and/or conditions will be expected to occur. (In the example above • (I) Within 12 months (2) among persdns 
under 17 years) 

(c) Specify the criteria, of acceptable performance and/or results by defining the minimum acceptable functioning level of the 
project. (In the example above * by 20%) 

Note: Complex projects may have more than one goal statementT*^ 

' (2)Identify implementation objectn^es for the project. Implementation objectives reflect major activities necessary to begin the 
project. They should be stated in the order in which they will happen. Describe how completion of cagh activity will be 
documented. For e^tample: 

(a) Space in which to conduct the project will be rented within two weeks of the subgrant effective date. A signed lease will 
' docunienWhis action. 

(b) Six new patrolmen will be hired within six weeks of the subgrant effective date. Signed personnel papers will document the 
accomplishment of this objective. 

(3)Identify performance objecth^es for the project. Pertbrmanep objectives indicate major behavior (activities) necessary to conduct 
tli^e project as planned. Each performancc^objective should incorporate, where applicable, specific behavior, the method or pro- 
cedures to be followed, time specifications and how achievement of the objective will be documented. Performance objectives 
should .answer the questions (I) Who? (2) What? (3X Where? (4) When? (5) How? (6) Under what conditions? (7) To what level of 
accept|ince? (8) As documented, by what? For example: j 

(a) The six patrolmen hired will successfully complete 120 hours of basic law enforcement training at the local community 
/ college within the first six months of the project. Documentation of course completion will be provided by the community 

college. " - ' 

(b) The Youth Services Center staff will teach 150 school personnel how to change unacceptable, illegal behavior patterns 
(delinquent bphavior) into acceptable, legal behavior patterns by using workshops to teach proper uie of techniques. A 
passing score of 70% must be made by personnel before receiving credit.Tor the workshop. Documentation will include^ 
attendance records, test scores and certification of course completion. 

^ Note: Technical assistance in preparing goal- statements, implementation objectives and performance objectives is available at 
regional planning agencies and the Division of Law and Order. 
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INSTRUCTIONS FOR PREPARATION OF SUBGRANT APPLICATION ^ PAGE 5 ITEMS 

U Methodt and ^ocedum: This item should describe the duties and tasks of each project employee and/or consultant necessary 
to achieve the goal($) and objectives discussed in the previous item. It should begin with a brief overview of the approach to be 
during the active Ufe of the project, followed by a precise definition of the actual methods which will be used to carry it out. This 
part will be necessary to project staff at the time of implementation. The information provided wUI facUitate judgements ^eprdlng 
the potential for timely, smooth implementation and project success. 

As a minimum this item should: * ..^ 

(1) Introduce the general approach through which the project proposes to deal with the specified proBlem. 

(2) Present in deUil each of the various steps or phases of the project. This pJesentation should include the following material: 

(a) An expUnation of how the project will progress to reach each implementation and performance objective listed on page 4. 

(b) A project schedule showing what wiU be accomplished at each phase and the estimated ^mc interval involved. Implemen- 
^ tation and performance objectives may be referenced by their number on page 4, 

(c) A description of how the proposed work wUI be organized and staffed {including a statement of hours per week for each 
person). . - 

1. describe precisely the responsibilities of the project director. 

2. describe the functions and responsibilities of all other project personnel. 

3. describe the exact role of administrative or advisory bodies concerned with the execution of this project. 

(d) The techniques and controls which will be used to manage each phase of the project. 

(e) The criteria which wUI be used to decide when to advance to each phase listed in the project, schedule in (2) (b) above. 

13) If equipment is to be purchased as a part of this project, describe its proposed use and the manner in which it will create or 
enhance capabilities. Provide justification for each major piece of equipment 

(4) Describe any special or technical assistance from outside your agency which wUI be required to complete the actual work and 
how this assistance will be utilized as an integral part of the project- 

(5) DeKribe physical facilities which are available and any additional facilities needed to complete the project. 

(6) Discuss anticipated problems associated with implementing the^pfoject and how these problems will be dealt with. 

(7) In addition to the basic requirements cited above, the following special requirements apply to applications for development of 
training programs. Each application should include: 

(a) A description of the method of identifying and selecting individuals who wUI receive the training, including criteria used to 
define those who are in need of the training. 

(b) A description of training materials proposed (attach an outline of the proposed curriculum). 

(c) A description of mcthod($) of presenting the material. 

(d) A description of training staff credentials. 

(e) A description of how retention of materials presented durinjf the training program will be measured. 
(0 A statement of the number and length of classes scheduled, plus projected starting dates. 

(g) An estinuite of the total number of trainees expected to complete the program and the identity of units of government 
expected to provide these trainees. 

(h) Any special credit, status, or certification to be awarded as a direct result of successful completion of the training program. 
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INSTRUCTIONS FOR PREPARATION OF SUBGR ANT APPLICATION PAGE 6 ITEMS 

19* Evaiu^tioii Dttigti: The eviluition design should describe the specific plans and activities which wiU be pursued to measure 
profirest toward and achievement of the goal statement(s) and objectives identiHeid ^on page 4* Final evaluation of the project and 
any consideration for continued funding will depend upon a well developed evaluation desi|n for documenting the achievement of 
the goal statement(s) and objectives. The evaluation delign should provide for and rely upon objective measures and judgimenU. 
Do not propoK a design which relies only upon the subjective judgement of project personnel, loal officials or adviiory groupt* 
Evaluation activities should be built into the wotk of the project rather than ""added on'! to it Evaluation actlrities should be 
Oflgoini throughout the project. ■:, * 

Thtjubject areu listed below should be addre&d as a minimum requirement for funding consideration: 
(Ohovide a general overview of the proposed evaluation design. 

(2) Denne how the level, scope or nature of the impact expected, as identified in the goal statement(s) and oWecthres on page 4, will 
be measured. Explain how accomplishments will be recorded. 

(a) What specific measuring devices will be utilized? 
X (b) What facts and figures will be collected? 

How will those facts and figures be used to show actions and outcome? *• 

(d) How will the facts and figures collected be related to the progress of the project? . ^ ' 

(e) How will these facts and figures be used to identify the strengths and weaknesses of theproject? 

(3) DeKribe how the information pined during the active life of the project will be used to make adjustments in the project and to 
improve the final prdduct or new services developed. 

(4) Identify project personnel responsible for completing evaluation activities described. Describe any specific outside technical 
assistance which will be required to conduct evaluation activities and the arrangements made to acquire it. 

{5)Explain the standards which have been established to determine whether or not the project will be continued at the conclusion 
of the subgrant period. 
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NORTH CAROLlNA-ExhiWt 2 



Subcrant Apptication Prbctssmg Form 



♦ ♦ ♦ EVALUATION COMPONENT REVIEW * * ♦ Sactlon D 



Reviewer 






Review D«t9 






• 





GOAL STATEMENT AND MEASURABLE OBJECTIVES 
Indicate: Adequate ( ): Inadequate (x): or Hot Applicable ( - ) 





' (1) 
Who? 


(2) 
What? 


0) 
Whtr«? 


(4) 
Whtn? 


(5> 
How? 


(6) UndtrWhat 

Conditiont? 


(7) To What Uvtl 
of Acctptanct? 


(8) As Documtittad 

by Whom? ^ 


Goal # 1 


















Goal # 2 


















Goal # 3 


















Implementation 
Objective # I 


















Implementation 
Objective * 2 


















Imptementartion 
Obiert»»e # 3 


















Performance 
Objective- # 1 


















Performance 
Objective # , 2 


















Performance 
Objective # • 3 












■ ^\ 







Application Description oft 


Adt* 

quate 


. * Inadequate: Explanation 


1. General Evaluation Design 




— : ' ' — ^ — ^ 1 


2. Methods and Procedures (for evaluation) ^ 






a. Measuring Devices 






Data Collection Procedures 




r — - 


3. Application of Evaluation lnform3tion v 






4. Staff (for evaluation activities) 






5, Standards forContinuation of Project 






6. Related Budget Items 






SUMMARY STATEMENTS: 
Goal Statement: 




Implementation Objectives (Inspection Factors): 




Performance Objectives (Evaluation Factors): ^ 





« state o< North Caroliiia • Division of Law and Order • Raleigh. North Carolina 
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NORTH CAROLINA-ExUbitS 



C, Department of Natural and Economic Resources 




Division of Law and Order 


Quarterly Subgrant 


r* ISOX Z/Oo7 £vtm 4 


Progress Report 


Rateigh, North Carolina 27611 CJ^tn^nV 




Subgrantee (Name L Address) 


Prolect Nnmh^r' 




Project Title: 


% 


0 



Date Submitted: 



Report PeriiKi: 
D^ite of Award: 
Djte of (mplemelhtation: 
Totil (^cJetal Award: 
Prugranri Citcgor V : 
Proicct Difectur 



.^.^^^:.,^..PkaseLaaw^^ thoroughly and succinctly in itemized attachments and additional documents. 

\ According to the ti^e criteria in the subgrant application (p. 4 k S). Is the project on schedule? 
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lU Have Special Conditions on the grant award been met? 

III. What specific problems or unexpected rcuUtihaveaffect^d^^ 

A. Financial . » • ^ 

, B. Proframatlc ^ ' 

What c6rrcct!ve action hai been taken this quarter, or is planned for the next quarter? 

IV* Whi^h of the three LEAAfoals is justification for thisproject? 

V, Itemize records bcmf maintained for evaluation pijfrposes, summarize data collected lh\% quarter, and 
^ specify the party responsible for its accuracy, ' 

VL Refer to goal statements, implementation objectives and performance objectives as iistedin the subgrant 
" tfpp/tor/o/» and outline progress made toward their achievenient%i$xiuarter, 

V\U " How many on-site visits were made during this quarter? Give date, parties Involved, and nature of visit, 

VIIL Additional comments: - ; i , 



NORTH CAROLlNA^Exhilrft4 



N. C Department of Natural and Economic Resources 
Division of Law and Order 
P. O. Box 27687 
Raleifh, North Carolina 2761 1 



Subgr^ntec (Name & Address) 



Blunt County Sheriff** Departacnt 
Bluntiylllc^ H.C. 



Date Submitted! XlVLllk 



Report PcriloJ 2 



Date of Award: 6/30/73 



Date onmplcmcntation: ^ lll/l^ 



Total Federal Award $5 , 837 • 00 



5AMPLE 
Quarterly Subgrant 

Progress fleport 



Project Number: 19-016-173-11 



Project Title: 



Detective Division - 
Blunt County 



Pf u|ram Category 



E-1 



Project Direetofj 



Sheriff Hopelong 



Please answer all questions thoroufhly and succinctly In Itemized attachments and additional documents. 
L According to the time criteria In the subgrant application (p. 4 & Sh Is the project on schedule? 
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IL Have Spcciiil Conditions on the grant award been met? 

ML What specific problems or unexpected rcsuUi have affected project actWfty? 
A» Financial 
B» Programatic 

What corrective action Itas been rakcn this quarter, or Is planned for the next quarter? 

IV. Which of the three LE AA goals is justification for this project? * 

V, llemi^c leccirds being maintained for evaluation purposes, summarize data collected this quarter, and - 
speeifv the party responsible for its accuracy* 

VI; Kcfer to goal statements, implementation ob|cciives and performance obleetives as fisted in the Siibgront 
appiicutmi and outline progress made toward their achievement this quarter. 

VII. How many on site visits were made during this quarter? Give date, parties involved, and nature of visit. 

Vni. Additional conimeiitL*. 



^ * - ifi Ml. Susie Smxtt 

Keport Cornpleled By _.. . 



1 itle tti>|iQ nml-Prnj»fit. Ani lYMt 

H;^!^ Januifjcy 10, 1974 
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Th« projtct !• on ichtdule except for the purchase of project equlpiient. The 
project ceUed for ell e<ptlp«nt to he purchased by Septeober 30, 1973. 
The only IteM of equip«ent purchased by that date were the caMra and 
electronic flaah. These two Items were purchased on Septeidier 30, 1973, from 
the capital Cera Shop at a cost of $152.09. I requested that the refin- 
ing Iteas of equipment be purchased by the end of January, 1974. 

tl^A^ conditions C-1 and 2 have been act. Special Condition C-3, requiring 
project personnel to receive 40 hours of training as approved by the Criminal 
Justice Training and Standards Council will be net by the end of the next 
be£j*»at ^^•'^^■^ Condition C-4 is a standard condition on all grants and is 

i'f*??'"/! « financial probl« with this project in that there are not 
sufficient funds for total operation of the detective division; the county 
wist assuae the cost of a vehlcU for the detective, aotor vehicle operation 

and various supply and operating ite... This* additional 
expense to the Sheriff's Departaent aay hinder their* ability to provide 
local ««tchlng funds for potential future projects. This situation will be 
discussed with the Regional Analyst to see if it can be reMdled in any 

B. Only two of the aight iteas of equipment have been purchased to date, 
therefore reducing the effectiveness of this project. The Project Director 
stated that all equipment Itew would be purchased by January 30, 1974. 1 
plan to follow up on this February 1» 1974. 

The LEAA Goal applicable to this project is "To Improve the Criminal Justice 
System. , 

A. Records being maintained for evaluation purposes are: 
1} Personnel Records 

2} Time and Attendance Reports 

3) Training Records 

4) Activity Repbrts 

5) Reports, of Crimes Cowitted 

6) Arrest Reports 

7) Reports of Crimes Solved 

B. Data collected this quarter is as follows: 

II records indicate that Tom Terrific was hired by Bluntsvllle 

Sheriff's Department July 31, 1973 as a Deputy; that same day Deputy 
Sam Slick was assigned to head the Detective Division. (The Adainistra- 
tlve Assistant is responsible for the accuracy of these records.) 

Deputy Terrific works .n average of 42 hours a week on the stcond 
N^ihlft from 3:00 p.m. until 11:00 p.m. Detective Slick works an average 

, ° hours a week; he works from 4:00 p.m. until 11:00 p.m. and is 
also available on request for investigation of crime scenes. (Same as 

, 3) Deputy Terrific is a probationary officer and has been recelvirig 6 
months of on-the-job training. The six months training will be coiq>leted 
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January 30, 1974; he will then attend a 160 hour Introduction to 
Pollde Science Courae to begin February 4, 1974 at Hew Hanover Tech- 
nical Inatltute. 

Detective Slick li presently taking an 80 hour Advanced Inveitlgatlon 
courae; he will coaplete thla couraa January 18, 1974. (Sane as fi.J 

- 4) Attached are aonthly activity reports for Deputy Terrific and 

Detective Slick. Deputy Terrlf^.c has spent all. of his tine this quarter 
receiving on-the-job training. Detective Slick has worked on 20 
cases during this report period. The Breakdown of these cases Is as 
follows: 10 - Breaking and Entering 
6 - Larcenies 

4 - Assaults v 
Of these 20 cases the 4 assault coii5>laints were determined to be 
unfounded, and 8 were cleared by arrests. (6 Breaking and Entering and 
2 larceny conplaints) therefore a total pf 12 cases were cleared. 
Sheriff Hopalong, Deputy Terrific and Detective Slick are responsible 
for the accuracy of these records. 

5) There were a total of 20 complaints filed this quarter. (Records 
Clerk is responsible for the accuracy of these records.) 

6) Fifteen arrests were made this report period. Of the fifteen 
arrest iftade, ei^t arrests were made for crimes committed th;^8 report 
period. (Same as #5.) 

'7) As previously mentioned, 20 complaints were fll^d in the categories 
of Breaking and Entering. Larceny and Assaults this report period. 12 
cases were cleared - the 4 assault cases were determined unfounded and 
8 cases were cleared by arrests. (Of th» 8 cases cleared by arrests. 

»6 were Breaking and Enterlngs and 2 were larceny.) (Same as f5.) 

The Goal Statement is that within twelve months after implementatloB the 
County of Blunt will realise a minimal of a^5X Increase in the number 
of crimes solved within the following three critte categories: 

Assaults - 7% Increase 

Larcenies - 5% increase 

Breaking & Entering - 8% increase 

This project wis implemented July 1, 1973; we are using July 1, 1973 as the 
beginning date for our statistical comparison. Each quarter we. include the 
number of complaints and cases cleared -for that period. We also t 
quarterly comparison to see if we are progressltjg towards our goal. (See ^ 
attached.) To date we are progressing towards our goal and feel tnat tne 
goal is realistic. / , 

SUMMARY OF STATISTICS TO DATE 
July * December, 1972 

tSumary not included in this Appendix.] 
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The preceding ■t«tl«tic*l data Indicates that the project goal is |>elng 
accoapUahcd. In 1972 thle deptrt»«Bt operated without a detective per 
se; each Individual deputy W to Investigate hli o»ra^«iaiei. Officer Slick 
oft«n aaaiatcd hie fellow deputies with Inveitlgation* because he bad a 
particular Interest In this area. Officer Slick was also a deputy and 
therefore had to perfota the various duties of a deputy. 

On July 31, 1973, Depuiqr Slick was assigned the responilblllty of startlnt 
a «etectlv« Division, . Since that date he devotes lOOt of his tlM (an 
^^•fl!**/!,^^.^'"" P*' Investigative work with et^hasis on cases 

in the tollowlnf three crine categories: Assault, Urceny ind Breaking t 
' *:heref6re assuoe that having a speclalited investliative 

officer In this departMnt has brought about the aforeaentloned Increases 
in coaplalnts cleared. 

l\ OMectives accoiiiplished to d*te» a?e as follows*. 1,2, 

3,4 and 6. Objective #5 has not been aceoaplished (see 1>; this objective 
should be acconplished by January 30» 1974. Objective #7 is continuous aiad 
on**goixiga 

^' II Objective #1 is a oOntinuous and on-going objective. 

2) Objective #2 will not be conpleted on ti«e. Satrolaan Terrific 
will complete the 6 aonths on-the-job training Jehuary 31, 1974; 
he will then attend the 160 hour Introduction to Police Science 
course at New Hanover Tech to begin February 4, 1974. " 

3) Perforwince Objective #3 was cofetpleted September 30, 1973} (a 
^ copy of the certificate is attached to this repbrt), 

4) Perfonwnce Objective #4 will be coiipleted January 18, 1974} (a 
copy of the certificate will accMpany the next Quarterly Sub* 
grant Progress Report). 

5) Perforaance Objective #5 1* continuous and on-going. Detective 

* Slick has worked with 45 eases in the areas of Assault*,. larcenies 
and Breaking and Entering* during the last three aonths; of those 
45 eases 30 have been cleared - 7 have been unfounded and 23 have 
been cleared by arfeats. " 

Vil. The only on-site visit aade during this quarter was the quarterly wmltotring 
visit Mde Oeceaber 15, 1973. At the tlM of this visit I distussed project 
activities and progress with Sheriff Hopalong and .Detective Slick. The 
Administrative Aasistant, Eleanor Rlgsby, sh<iM«d ae all project records and 
Quarterly Subgrant Monitoring Report. 

VIII.Additlonal CowMnts ' 

This project Is progressing on schedule and is accoapllshlng the project 
5?'^:..f**** •■t»bli«hed for this project, the Blunt County 

Sheriff's Departaent was not eure how realistic it was. To date this 
project is accoapllshlng its goal and the Sheriff ♦* Departaent Is Very 
pleased with the results. The Sheriff and Detective Slick have discussed 
aoving two other officers Into the Detective Division in order to have an 
Investigator available a alniaua of 16 'hours a day. They are planning to 
subait an appl^atlon for 1974 funds to assist th*a in establishing this 
Division. ^„ 
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! \ NORTH CAROUNA-ExWbit 5 

[■ . 

TH1N6S TO LOOK FOR WHEN REVIEWING A QUARTERLY REPORT FOR AN INVESTIGATE 
tJNIT PRDJEpT: 

Kain things I will be looking for arc: 

1) I» the Lroject living up to the goal», objective and evaluation design 
rfectioi^a of the application? What records arc being maintained to 
determine how and when Implementation and Pcrfortoauce Objectives are 
being accomplished? 

2) Is the (quarterly subgrant progrcsB report coBq)lete and factual? 
ITEMS TO inkuDE IN A QUARTERLt REPORT FOR AN INVESTIGATIVE UNIT PROJECT: 

1) How and! when was the project Implemented? 

2) Were ndw off icers hired or were present officers promoted into this 
specialised area? Were officers hired to replace the officers that 
were promoted? 

3) How were these officers selected? 

4) What is the background of these officers that were moved into this 
specialized area? Have they had specialized training br will they 
receive specialized tJraining during the grant period? 

5) What are the spiecific duties of this squad or unit? 

6) li One person designated as the ^quad leader? 

7) What hours do these officers work; what shifts? 

8) Each squad member should prepare a monthly activity report indicating 
how many '^tases he worked on - what kind of cases and how many were 
solved - training he has received - other activities, 

9) Is base line data available? What does this data indicate? A 
quarterly consparison should be made between the situation that existed 
and the present situation, 

10) Is the goal or objectives unrealistic 0|r nonworkable? If so» a 
revision should b# made* 
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Qu«rt«rly f rogrttt UipoTt \ 
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PIOJICX mMBEH: 

OAII: 



PEKSOM COWTACriD: 
ACTIOH TAXmt 
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B. Exampl#s of Monitoring Systems 
Colkcting Quantitativo Data 

Recently, several SPAs have developed monitor- 
inf systems utilizing a standardized reporting form 
designed to produce quantitative data on specified 
events. In some cases, the events and measures to be 
used are specified for all projects of a given type. 
In other systems the specification is on a proje^ct by 
project basis. Examples given here are for systems 
operated by the Connecticut, Oklahoma, Ohio, and 
* South Carolina SPAs. The Indiana SPA is also 
developing such a system* 

1, A Systsi^ With Prtipwlfiid Claim of 
Ivmfs to bo Monttorsd 

Connecticut** Continuous Evaluation Data Col- 
lecting System (CEDCS) serves two functions. It 
provides baseline data for anticipated in-depth 
evaluations and is used to monitor-existing projects. 
As a monitoring tool it is one component in a 
system which -^so includes a monthly narrative 
rtonitoring report completed by RPU t)lanners after 
a site visit to a project. Exhibit 1 is a list of the 
topics planners are to address in their reports* 

The CEDCS provides the Evaluation Unit with 
data on projects through the Evaluation Progress 
Report, a two-part report which is completed quar* 
terly by project directors* The first part of the repprt 
contains data on prespecified measures covering 
background information, input and output items 
and project eflfcctivencss. Exhibit 2 is the list of the 
items covered on most reports. When appropriate, 
t a few additional items, such police or court statis- 
/ tics, arc included* When each grant award is made, 
the Evaluation Unit designates specific events, called 
data clciiients, to be measured in each of the four 
classes of events. Those *'data elements are specifi- 
cally tailored to be responsive to each project's 
special performance objectives and activities/'* 
Based on the project description in the subgrant 



application, the evaluator making Jhe assignment 
determines what data elements are appropriate for 
a project. Exhibit 3 is an example of a CEDCS data 
collection form. 

When completing the quarterly Report, the project 
director also lists the objectives of the project 
whether* or not they were stated in measurable terms 
in the grant application* He does not have to relate 
the measures he states for any projfect element to the 
project^s objectives. This is, in part, what the evalua* 
tor at the SPA does when reports are filed, 

**Part two of the Progress Reprt Form provides 
the project director an opportunity to analyze and 
qualify the data presented and to delineate prob-^ 
lems and expectations he encountered throughout 
the quarter*'' ^ - ' ^ 

Fifteen (15) days after the end of the fiscal quar- 
ter\hc reports are due at the Regional^ Planning Unit 
(RPU) where the planners (who are famifiar with 
thfc projects since they are also responsible for the' 
monililv sit^ visits and narrative monitoring report), 
check mhibfor accuracy .and forward them to the 
Evaluation USnit at the SPA within a week of their 
receipt* A 

The Evaluaion Division then analy^cs the Reports 
and prepares la report with comments Jor each 
Project. A eody of the Evaluation Division report 
is given as Exhibit 4. A copy of both the project 
director's Progress Report and the Evaluation Divi- 
sion comments arc sent to a SPA Staff Planner, the 
Grant Fife and the Regional Planning office. It is 
the responsibility of the regional planner to show 
the Evaluation Division Comments to the Prdject 
Director. After two quarters, the Evaluation Division 
reviews Evaluation Progress Reports and RPU mom» 
toring reports and makes recommendations to plan- 
ners about changes that should be made in some 
projects that are requesting refunding. 

Quotes from documcntatlfon providcil by ConBcctieut 
Pl»fttting CommJltcc on CtMmt Adminbtfatiion. 
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Areas to be addressed regional planners in their monthly monitoring reports. 

I. I^or all monitoring reports whether on-site or telephone, the following areas should be addressed: 

A. Programmatic 

a. Is the project proceeding toward its stated objectives? 

b. Is the project utilizing the methods and procedures enumerated in the application to achieve its 
goals and objectives? 

c. Is the quality of staffing in conformity with the application? 

d. Is the applicant keeping adequate records of his activities? " 

e. If staff training is involved, it is in conformity with^e application in its quality and quantity? 

f * If the program category calls for a specific requirements to be fulfilled, has this been accomplished? 

B. Financial 

a. Is the grantee conducting his affairs in conformity with his budget? 

b. Is the grantee drawing down funds at a proper rate? 

c. Is the grantee maintaining adequate financial records? 

C. Evaluation ^ . 

a. If the program category has called for an cvaluatipn component, is the grantee maintaining records 
on a continuing basis for this purpose? 

b. If the Audit and Evaluation Division has set up specific methods for the collection of data with 
the grantee, is the grantee conforming to the requirements laid jdown by that division? 

c. Are there any variations between the Project Director's Evaluation Reports submitted to the 
Connecticut Planning Committee on Criminal Administration and your observations? 

Additionally, review the milestones and note the position of the project at the time of the report. 

II. Any significant problems which are encountered should be noted together with recommendations to 

alleviate them. . . 

Positive project accomplishments sh^ 

An overaJnubjecHve ev^^uation of theproject's achievements should also be included. 

it 
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CODING STRUCTURE 

l.O Backgrouiul Dafa ^ 

1 . 1 Tarfier PopuUaion % 
lAA Profile characierbtics of population and/or area problems (Nature and Scope) 
1 .2 A vtiihible Scnkes Index 

1.2. 1 Specialized target agencies in area ^ 

1 .2.2 Quantity and Ty pe of Beneficiaries in above agencies 
2.0 Input Data 

2. 1 Staff atul Resources Committed 

2.1.1 Type and Quantity of staff personnel 

2.1.2 Type and Quantity of specialized p^rsonneUs^ilablc to project (i.e., consultants, volun- 
teers, students, professionals) 

2. 1 .3 Boards, agencies, planning description and participation committee 

2. 1 .4 Equipment, buildings, land available to referral and retrieval agencies 

2.2 Staff Activity Measnres 

22 A Percentage and/or manhours of major staff time expenditures by category 

2.2.2 Program phase activity units 

2.2.3 Meetings with community leaders relevant to project 

2.2.4 Caseload Data 

2.2.5 Referrals retrieved by quantity and type 
3.0 Output Data 

3.1 Beneficiary Description 

3.1.1 Active/Inactive Cases on File 

3.1.2 Profile characteristics " 

3. 1 .3 Referral Problem(s) — ^Nature and Scope 

3.1.4 Identified Problem(s>7-Naturc and Scope 

3.2 Service Provision Index 

3.2.1 Program phase participation/type and quantity * 

3.2.2 Type and duration of client contact 

3.2.3 Specialized services/techniques 

3.2.4 Referrals by quantity and type 

3.2.5 Type and duration of follow-up on referrals 
4.0 Effectiveness Data 

4.1 JShort-T^rm Impact 

AAA Program phjfttc service completions and terminations 

4.1.2 Consummation of Referrals ' * ; 

4. 1 .3 Frequency staff and client attendance of prograni phases 

4. 1 .4 Improvement of Problem Identification 

4.2 Long-Term Impact ^ , ' 

4.2.1 Reduction of specialized problem(s) 

4.2.2 Improvement of CIS coordination 

4.2.3 Improvement of program clients by type and quantity 

4.2.4 Longitudin al progression of client receiving services 

4.2.5 Iniprovcmcnt of program capabilities 
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CONNECTICUT-Elxliibtt 3 



.EVALUATION PROGRESS REPORT 



PROJECT TITLE: Crl«e Prevention. Bureau 

GRANTEE; City « 

PROJECT DIRECTOR; ChleT 



PROJECT ADDRESS: Pollcfe Departaent 



PROGRAM CATEGORY: 2.2, 
GRANT NUMBER; A-73=oo-.oqooo 
AMOUNT; $20,140 
TELEPHOTIEr^ 



City. Conne c ticut . 



PROJECT OBJECTIVES 

To Inform and educate the public with police practices and pollcle* In dealing with 

1 . and reduclni^ atreet crime -' " ' 

To make the public aware of police department law enforcement teaponalbllltlea 

2, through the dlaaemlnatlon of Information concerning crime pryvepfion t, mttht^dm 

3^ To dealgn and carr^ out projecta In the crime prevention area on an experimental baala 

,, BACKGROUND DATA 



;(Ide 



EVALUATION MEASURE 



4 

X 



burglwles-clty-wlde 
burglaries-target area 
Iresidentlal 
#cof9Bmerclal 

arrests for burglarles-clty-^wldt 
arrests for burglaries-target area 
citizen complaints /total area 
citizen complaints /target area 
complalnts/burglarles/target area ' 
complalntct/pers* violence/target area 
complaints against police 
personal violence crlmes/clty 
personal violence crimes/target area 
legislative documents concerning target activity 
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imi DATA 







:^ 


1 


2 


3 


4 1 


c 


ODE, E 


VALUATION MEASURE 


p 


A 


P ' 


A 


P 


A 


P 




s 




# professionals oir staff 
#full-tl«e police officers 
fclvlllans 

# volunteers or sub-professlonals 

# covttunlty pcograms designed 

# cottBunlty programs held 

# 1>rochu^e8 prepared 

# brochures passed out ^ ' 

# brochures to target area 

# seminars/workshops with conmiunlty 

# novles/slldes shavm 

# meetings with community organizations 

# meetings with public organizations 

# meetings with private organizations 

# requests for consultant assistance 

# experimental programs designed 

# programs In operation 


















^ 




% 


> 
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OUTPUT DATA • 



CODE 


EVALUATION MEASURE 


1 


2 


3 


4 


A 


A 


A 


A 




# coBipiainti received/target areA 

# coaplalnti burglaries 

# cleared coaplainta 

# uncleared co«plalnts 

# conplalnts /personal violence ^ 

# cleared cooplaints 
#, uncleared complaints 

# residence hones Inprovlng locks 

# conniirclaXSestabllshaents laqptoving idcka 

# engraving machines in operation 

# citizens reporting crimes 

# citizens reporting burglaries /target aria 

# citizens reporting personal violence criMs/target 

# connerclal establlshaents reporting burglaries 

# comerclal establlshaents reporting personal 

violence crimes 

# citizens Involved In meetings 

# citizens In workshops/seminars 

# citizens viewing novles/slldes 

# citizens Involved In other planned activities 
1 legislative changes made/qtr, 

* 

-J* 

* 




o 
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EFFECTIVENESii DATA 



X 



CODE 



EVALUATION MEASURE 



3 

"A" 



In 



reduction of burglary rate 
reduction of burglary rate In target areaj 
-in realdigicei" 
-In conerclal establlslimenta 
reduction of peraonal violence crlae rate 

takg#t area 
repetition crlaea on vulnerable platiea 
vulnerable placea Identified 
offender* a patterna recognized 
eatimated of connunlty making any changes 

advocated by ptoject 
previoualy unreported crime now being reported 



FOOTNOTES (On Background , Input, Output or Effectiveness Data) 
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PROJECT NARRATIVE 

biscusston of Administrative Aspects of Progranrolng (Backgrouwl Data, 
Input Data) 



2. Discussion of Operational Aspects of Progranming (Output Data, 
Effectiveness Data) 



3, Expectations For the Future 
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Cow en ts 



PROJECT NARRATIVE (contlnutd) 



CSigr>ature of Respondent} 
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CONNECTICUT- Exhibit 4 



EVAIUATIOH DIVISION PROJECT COWENTS 
' (Based on CEDCS Reports) 



PROJECT TITLE: 
GRXNTEE: 



PROJECT DIRECTOR: . 
PROJECT ADDRESS: ~ 



RESPONSIBLE EVALDSnJlH" 



PROGRAM CATEGORY N0. : 

GRANT NUMBER: ' 

AMOUNT: '* 
TELEPHONtr 



DATE OF LASt PROJECT .SITE 
VISIT: 



PROJECT OBJECTIVES 



1. 
2. 
3. 



1. PROGRESS AGAINST STRAGEGY | 

A. Administrative Milestones (Perfomwince Objectives) 



B. Program Development (Background Data, Input Data) 

C. Program Component Implementation (Output Data) 

0. Anticipated Impact (Effectiveness Data) 

1. Impact on CJS 

2. Impact on Specific crime problem 
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strengths of Progranwing (Results of specified outstanding large or 
otherwise notable subgrants) 



Weaknesses of Progranmlng^Descrlptlon or evaluation of problems 
encountered- In subscription, implementation, monitoring, evaluation 
or applications of miscellaneous federal or state requirements to 
this program) 



Impact of Results Upon Progif*amm1ng (Sample of use of results of 
this program In future plan development and implementation) 



GENERAL COMMENTS 
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A Syst«m fh«t M<»nif«r« Totks PrtiiMcifbil 
by EcKh l!r»ji<f l>iir«ct«r 

The Oklthomt SPA recently developed t system 
to provide an evaluation data ba$e and to motiitor 
the performance of projects quarterly on the basis 
of planned versus actual achievement. The subgrant 
application requirts*a clear statement of the problem 
and supportive data» measurable goals, objectives 
i^nd project activities* Exliibit 1 contains directions 
for completing that part of 4he application. The sub* 
grant application «lso includes a copy of the Quar- 
terly Evaluation Report on whfch the applicant is 
instructed to list the tasks developed in the project 
plan and to project quarterly levels of achievement 
for each. Exhibit 2 i% a copy of the report and 
Exhibit 3 is a copy of the instructions to the 
applicant. (See pages 57 through 59) 
^ Those pages of the application are reviewed 
by the evaluation staff prior to consideration for 
funding. If they are found to be inadequate, m 
jcoriferencc is held with the applicant to negotiate 
the deficiencies* 

Besides necessitating improved project planning 
on the part of the applicants and providing a; sound 
basis for monitoring and evaluation, the review com^ 
mittees of the SPA advisory Commission have relied 
ori^ these pages of the application to Jearn exactly 
what the project intends to accomplish and the 
methods, they .intend to employ in making their 
recommendations on funding to the full Commission. 

9* A Sytftm Using Pr««formaftaii Insfrumanff 

Ohio's evaluation/monitoring systent serves the 
two-fold purpose of enabling staff to monitor projr 
ects with Jittle dittci contact and supplying data 
with which evaluations can be performed. Monitor- 
ing responsibilities for implementing this system are 
divided between the SPA and RPUs. The RPUs are 
combination city*county planning agencies located 
in the six; largest cities in the Slate* Monitoring in 
the State's six RPUs is handled at the regional ofliccs. 
The SPA monitors those projects in the remainder 
of the State and in the state agencies receiving LEAA 
funds* (See pages 60 through 69 for Ohio Exhibits 

t5c core of the Ohio system is a set of data col* 
lection packages Specially designed for 23 program 
areas. Exhibit 1 is a partial list of program iypts and 
their assigned evaluation/monitoring package. Each 
package contains a data collection Instrument which 
the project director completes quarterly as well as an 
introduction to the system tot the project director, a 



list of definitions of terms used in the instrument, a 
set of questions which give the proiect director an 
opportunity to explain trends in tne ^uta, and instruc* 
tions for the analysis of the instrument. Exhibit 2 it 
\n example of one evaluation/monitoring package. 

When a grant fward i% made, an SPA planner 
with monitoring and evaluation responsibilities it 
given a brief 'project summary and, based on that 
summary and a pre-determined cross reference list, 
at least one instrument is assigned to the project for 
reporting purposes (occasionally a large project is 
assigned more than one instrument). A data sheet 
in matrix form and a questionnaire are filled out by 
the project director at the start of a project specify- 
ing time-^phased project goals and .baseline data, 
when it is ayailable« against which performance 
will be measured. 

Since monitoring of projects m RPUs it the 
responsibility of the RPU evaluation' ttaffs, those 
staffs also assign data collection instruments to proj- 
ects in their regions. They occuionally obtain SPA 
permission to make changes in instruments used by 
their projects if it is agreed that those changes would 
enable the instruments to reflect their projects more 
accurately. The regional evaluators retain the instruct 
tions for analysis so they^can^ perform analyses when 
the reports are submitted to them. They do not send 
to the SPA copies of the monitoring reports sub* 
mitted to them but may use those repotts to prepare 
a summary report for their advisory boards Exhibit 
3 is an example of a summary report prepared in 
one RPU. Information on monitoring performed by 
ftPUs is sent to the SPA only when the State agency 
needs such information for plapning. 

Analysis of reports from the non-metropolitan 
districts in the State is done by the planners in the 
SPA. The analysis is based on data supplied by the 
subgrantee^ including the projected goats which were 
determined by the project director and, when pos- 
sible, on baseline data for a period preceding the 
start of the project. After analysis of a report a 
Ptoject Evaluation Summary is prepared by the plan- 
ner. Exhibit 4 is a copy of that Summary and the 
instructions for completing it. 

In addition to the component just described, the 
SPA uses an additional method tof monitoring the 
projects in the non*metropolitan^ districts and State 
agency projects. Field staflt teams who are assigned 
to the non-metropolitan districts, prepare a Project 
Monitoring Report on each project. Exhibit 5 is 
a copy of that Report. 



OKLAHOMA-Exldbit 1 



17» Prtitct Hm whI liyyif H ift Dtfi: Tht mforfnotiond ittms foond on this po|f or« t sstntral to thi cons»4trQt(on of tny «pplctti«A ftr 
subftcmt fun^s. Wtxlt cirttiin ittmt moy roquk't mort ^oborolion thon athtrs^ no ittm should b« pmittod. tf ''Hot Afi^ktW li fffm 
1$ 0 rtsfonst, on tx|»^lon shoutd bt bci^td. Plfos« follow instructions ctoso(y» btfnf cartful to incNo i4l rt^mttd ( nftriw i litii^ 
This sKtion of Ittf o|»^otion is tht bosil not only for considrotlon of your $ul»gront rtqutst^ but for Itttr tvtlutttfn tf ytur f9r- 
fofnwncti 



A* Whtif Tbt iN'tWtinT V > 

This Mctfon shoutel includt a britf but compfttt pi(utrt of tht dtltnqutncy. crimt^iinttd or proctss*oritnttd proWtm ytM h«#t to 
oddrtss with this suinront- Tor txomplt« whot ort tht maior crimts^ whot is tht juvtnilt commitmtnt rptt for tht orti to bt ftrvtdi 
what is tht coffiporotiivt cost of Instrtutionoliiotion os o^std to susptndtd stnttncts or perolts whot fs tht tumovtr ntt ftr |Mlkt 
off»ctrt? Such ^nformot^on os workloods, populotion. dtmogrophtc doto. ttc. moy bt includtd htrt* » rtltVMt* Tht doto (NTfvldtd htrt 
wiN of courst ifftr occording to tht typt of projKt tnvol^. Tht progrom dtscription in tht Stott Plem shouW bt ctnsulttd fir 
sptctfk informotiton rt#rtd» 

i. WlittNYttlHptTtOtAbtttltr(ii»HiimMttt((t)] 

This SKtion should contoin ip%t^Uc, mtosurabit gocis m dtoling with tht probltm outtintd obovt. F^or txaniptt^ 

Hr CHmt SftiHk fn^HHt 

Antkipottd chongt in tht crimt rott, orrtst rott. solution rott. ttc. 
ftr INrilt^pMiMy ^rtftntltii IPr^jtcftt 

Numbtr of ptrsons to bt providtd strvkts ond sourct of rtfirrot ontkipojtd chongt In odjud»cotlon rott (ond/or chonft in crimt 
rottv H ovoiloblt)^ ontkfpottd numbtr of ptrsons to bt prtvtnttd from invotvtmtnt in tht lustkt proctssti^i ontk^ttd numbtr for 
whom on olttmotivt to odjudkotion is to bt providid. 

Hr AM mi l»m9t CtrrtctftM Trtttmtnt rrt{f ctis ^ 

Antkipottd numbtr of ptrsons for whom services will be provUded^ ontieipoted chongt rn rtckiivism rott of ptrsons strvtd by tht 
progrom. 

Ptr TnMlf hrt|tttt} 

Numbtr of ptrsons to bt trointd^ of troming to bt of ftrtd (contents skills to bt oc()uirt<l. ontklpottd ottihKk chtngis; 0|h 

proximott omount of timt to bt spent in tfoimng. , 

Nf twyliymiHt fN^iftcti [lucftdhif bKrttf tug rtg vltr mtnftwtr ttd firttniilitpt]t 

Numbtr of ptrsons to bt hirtd.^ kind of job to bt performed, anttctpottd wirkiood of persons tmptoyed under tht project ond ontk^pottd 
improvtmtnt in workloods ond/ior working conditions in the of fkt/ogtncy. 
Ftr It i eifil i md Htmltif ^reiectt: 

Numbtr ond typt of projects to bt undertaken onticipoted impoct on cHmtnot justice process, or on o specifk crime or system proMtm, 
Ftf All Prefectii ii 

Antkipoftd cost/btntfif factor (Exompit > Compofotivt cost per offtnder treoted^ cost per troinn^ cost ptr criminol cost brkf ed by 
Itgpl intemi^i ttcj^ontkipottd dtsirobit or undtsirobit sidt tf fKts oi o rtsult of tht projfKt. 



/ 



MpptfTlTt 



1/* Mttf Hm Ami S4f^« Ml (etfltkwt^) 

C l««1*rlltwWlltttOtHT(itefttiidtt«ki)T 

This section should inckidt a dtKriptton of spKific stops to bt token to «compliih the goolts) of tht project. ThMt steps shtuM bt 
i^uonttfitd if posslbit ond listed in numerkol order Informotion on sptcifk tosks thot will implement the obovt listed stt0 should 

ft^tow. Thtst tasks should ^so bt quontif»ed if possible ond fisted in numtricol order (use oddit^onol sheets if ntctssory). 

-J > 

NtwifmlfttMCe«iMeMlttfiMet.efHe^ 

If tht progrom deKription in the Stote Plon under whkh you ore opplying for funds has ip#ckil conditions, how hove ypu comjil?ei with 
them* or how do you txpKt to comply with them during the project covered by this subgront? (e g ,, i^depth evokwtion, iptckrf 
rtstkrtifafis. dtcrtasing funding, ttcj 
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APniCATK)N rot SUKtANT 



K OUARTWY CVAUMTKM mOkX 



CMt>f(ntfft ftctor 



. Sowct of Momtfhn . 
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SuwHory 


4th Quorttr 


3rd Quortor 


3nd QMTtor 


1st Qutrtor ^ 


Expocttd 


Actuoi 


Expoctod 


Actuol 


Sxptctod 


Actuol 


> Expoctod 


Acnraf 


Exfoctttf 


Actuol 
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^ Qitifttf ^ Which Sp#CNi( 
utnviiwni ififf iwtT 
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Stttt souft*(t) in mpM^ of mrrkmit) ttirttd i^ovt. 
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Nf mm 1f»n tot coit/b*nifit f^tor f xists os <kfin#d in ^ imtructioot tnlHi bod tf iMitft mii«n I on^ "F^ All rrp»*ctii/* pjimt 
nuinbor •«! Ii$t Hch fidtr itfWtMy. ^^...^^^^ • * 

P'Mitalliiv €ii#f# iff* IwyMftd by Ihrt^iiitt ^ 

fkm$ $tttt. in 0»t iwnmory mtkn, ^Jon (numHr) »f cwtf ond/pr liMMntt listtd obovt tiwth w hkh tfirs Mttt 
M «rK% til tt«n p$^\m frtor wti ttvtn obovt. ptm* Hit JnMmlVJ Also fiit tho tx^ttd nwr*^ by <»uortt«i 
. 1t»tfifyfointt>oiu«mofyiK«onwMnotnoc«sorilyNt^ 
jMVoniloi for four f|MOftir$ you wogW ihow ^ 
(imniloo ooch <»uortor yo«r totol wogW lo 100.) 



li MM SfoM ConAMoM Wort Mitt 

Wrt»o^*yot" \n iho i|Uort^ ^ whkh j^kW conitiom oro oxpoctMl to bo mot. If tho profrom ckKriptioft in tho $toto f Ion doot not 
Kovo 1^ condftiow* or if Itioy hovo olroo<ly boon mot, $how H/A in tho tomnwy mtion. 



m tt»o sMiimory mtion of e #vo tho totd numbor of om^oyooi yoM onlki^o hcrvinQ hirod by tho ond of tho iKojoct poM.. In e (14 stoto 
tfio oxpocfod numbor of mon hour j to<bo dwotod to tho proltKt. For ooch <|i«rtor stofo tho nwmbor of ompjoyoot and mon houn for thot 
tiuirtor* 

ttowbor ol Vt<on m i i > i(h)liim Hooro fhnHi to Mottt 
Troot in tho lomo monnor OS c ond c (14 



ms - tAMt 



Nm I, ioction C rooMOiH thot you itoto Hiocific itofn to bo tokon to »otvo tho lyoWom ond olio ipKifk fttks to bo porformod to Smpfomont tho, 
ttopo. Hotio nuwbor tho ipo<Hk sfopo on po|o 4. mtkn C ortd tHw, in tho iwnmory mtion undor tho corroi#ond*n| numbor. Kit tho rot^ti 
(tuwttrfWd. poiiilo) oxpoctod ot tho ond oftfio projKt. Aiio lilt your o)tpoctod qgoHofly roodti. If it ii not pof iM 
^MTtotfy rotoHi, writo; * ^ ^ dur^nf ^h you plon to implomint ooch itip ond in oil wccoiilvo ^ori dunnf irhkh tho ifop 
mil bo o#orottyo. 



Tfoot in tho lomo monnor oi Stopo» 
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OHIO^ExhiUt 1 

EVALUATION , INSTRUMENT 1973 CROSS REFERENCE ' ^ 

A. Law Enforcement - * # 
1- Recruiting^ Testing of Law Enforcement Personnel 

a. Rccruftment ; .EQ^!5 

b. Testii)g ...... . ^ . .\ .............. . . . .EQ^19 

2, Training of Law Enforcement Personnel EO-'^O 

3* Construction & Improvement of Law Enforcement Facilities T 

a. Construction ^ EQ-4 

b. Renovation ! ! * EQ-17 

4. Manpower Utilizatidn, Planning & Management Assistance 

a. Internal Administration ^ EQ-8 

b. Line Support . , . , * EQ^ll 

c. Management Information > .EQ^12 

5. Legal Advisors , 

a. Investigation EQ-9 

b. Legal Aids .EQ^IO 

6. Science & Law Enforcement 

a. Apprehension, Detection & Scientific Investigation EQ-1 

b. Identification ^ EQ^7 

7. Radio Communications : EQ-1 

8. Special Enforcement Problems * ^ 

a. Apprehension, Detection & Scientific Investigation EQ-1 * 

b. Civil Disturbance ; EQ^2 

c. Deterrence ; EO-6 " 

d. identification \ ^ £0-7 

e. Line Support ; ^ ^ .EQ-11 

B. Prevention and Diversion 

1 , Special Security Programs for High Crime Areas 

a. Apprehension, Detection & Scientific Investigation EQ-1 
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STATE OF OHIO 

X>HN J GiLUQAN. Gov«rnor 




OHIO-ExMMt 2 
Evaluatioil/Moiiitortiit IiMtnuneiit 



'•DEPARTMENT OF ECONOMIC Am COMMUNITY DEVELOPMENT 



DAVID C. SWEET Dirtctor 

TO: Subgrantees 

FROM; Alphonso C. Montgomery, Deputy Director ^ 
Administration of Justice Division 

SUBJECT: Evaluation Instruments 

Attached please find evaluation and monitoring instructions and 
forms. These instruments have been selected for use in your s ^ 
project according to the information found in your project . 
application. We feel that the information requested can be as 
helpful to you ^n managing your project as it is to AJD in ^ 
justifying the Continual flow of Federal assistance to Ohio. 

Realizing the weight of your other administrative duties, we have ' 
tried^ttj limit thp scope of this request to only the most basxc 
measures of productivity. By identifying these measures, we nope 
to gsi^iti the r.03t x>ertinent information in the easiest manner. 

Please reed the introduction ?nd definitions and review the data 
sheet and questionnaire upon receipt', so that you may begin 
gathering the required information. Baseline (if required) and 
ffoal data should be submitted with the fd^rst quarterly report. The 
baseline and goal data contained in the initial report shfuld not 
be changed in later reports unless it was found to be inaccurate. 
Any change in following reports should be accompanied with a letter 
of explanation. 

Evaluation reports for your project are due at AJD on a quarterly 
basis (i.e., every March, June, September and December). You are 
allowed ten days following your reporting month in which to return 
the evaluation report. Please send your. report in with your 
monthly financial reportw^Failure to comply can result in a delay 
in the delivery and proce^g'ing of your funds. If for some reason 
you to Submit Ihe ^.alu^li*.** reports on the vcqui-Cu 'laic 

please inform this office *as soon as possible. If you have any 
difficulties. in understanding or obtaining .the required information, 
please contact this ioffice through your^JD field representative at 
(6141 /166-5126. 

^uaAxwudi; vvu. , aata which may prove helpful to us is 
welcomed. Thank you for your consideration. 

ACM: pah v 
ATTACHMENT 

ADMhJl *TfiA1h>« OF JU5TICC OlVljiO^ • KO' iojl . iX'i UVdJlS OHIO 1 ^ MO * ol4 4r.c, 
* OM^JUDiAIAN uihECT LINf * Toil ^ tn Ohio s eOO ?8? »0«5 • Oul^f S»at» 1 -BOO'B^I^^ iO? 
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Instructions for Compfotino AJD EvaiMotion InstrMmontf 



EQ 10a 



— LEGAL AND (DEFENSE) 



Projects covered by this questionnaire are nor- 
mally concerned with the provision t)f legal defense 
to the defendants thcmsejve§. 

The quantity of legal advice provided is measured 
in terms of man/days and cases. 

The purpose of giving you this questionnaire at 
the start of the project is to enable you to develop ^ 
sources for the required information if they do not 
presently exist. Realizing the weight of your other 
administrative duties, we have tried to limit t|ie 
scope of this request only to the most basic mea- 
sures of the productiveness of your project. These 
figures can be as helpful to you in managing the 
project as they are to AJD in justifying the con- 
tihlied flow of Federal assistance into Ohio. 

The attached data sheet and questionnaire should 
Jbe filled out at the start of the project, giving close 
attention to the goals of the project and the baseline 
data with which its performance will be compared. 

Definitions 

L Number of Indigent Defendants in Jurisdiction: 
Legal counsel; given to individual defendants 
is being evaluated here. This figure should be 
the total number of complaints filed by the 



police or the number of indigent criminal 
defendants charged in the court(s) or juris* 
diction of the pioject per week or per month. 

2. Number of Indigent Defendants Given Legal 
Counsel (ACTUAL): Either cases or defen- 
dants must be used continuously throughout 
for comparability. This is the number which 
received some legal advice. This number should 
be a portion of #1 above. 

3. Number of Indigent Defendants Given Legal 
Counsel (GOAL): Similar to #2 abovc^ but 
established as a goal for the project. Again, 
this may be either cases or indigent defendants 
depending on the project, but one or the other 
must be usedncontinuofusly. 

4. Number of Days of Counsel Given (ACTUAL): 
The number of days which attorneys spent on 
indigent defendants in line' #2 a^ve, a day is 
normally in the vicinity of 6-8 hours. 

5. Number of Days of Counsel Given (GOAL): 
Similar to line #4 above, but established as a 
goal for the project prior to commencement. 

6. Baseline Data: Data covering a period equal 
to the project period, and preferably immedi- 
ately prior to the project, if applicable. 



Quarter I ends ^ 
Quarter II ends 
Quarter III ends 
Quarter IV ends 



LEGAL AID (DEFENSE) 



Base Data 



Quarter 
I 



Quarters 
I ^11 



Quarters 

ijijii 



Total for 
Project 
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Number of Jndigent 
Defendants in 
Jurisdiction 



Number of Indigent 
Defendants Given Legal 
Counsel (ACTUAL) 



3. Number of Indigent 
Defendants Given Legal ' 
Counsel (GOAL) 



, Nttmlier of Days of Counsel 
Given (ACTUAL) 



5. Number of Days of 
Counsel Given (GOAL) 



6. Baseline Data 



1. What changes, if any, have you made to the 
goals of the project since its inception? 
Explain wliy you made these changes. 

2. What changes, if any, have you made to the 
categories of cases receiving legal counsel, to 
the methods of counseling? 

' Explain why you made these changes. 

3. Do you feel the additional efforts expended in 
this project justify the results? 

Why or why not? 



4. Should the project be permanently incoporated 
into the system? 

What modifications are necessary? 

5. What other informiation (perhaps anticipated) 
has resulted from this project? 

6. Do you think the quality of legal counsel given 
can measured? 

If so, how? ; 

7. What suggestions concerning evaluation of 
similar projects can you make? 



How to Anolyzo Ugal Aid Evoluation Form 



1. Dka Check 

fhis analysis should be performed on each report 
before other analyses are begun. If the data proves 
to be incomplete or inaccurate, the subgrantec should 
be notified to make revisions so that the analysis 
can continue. - * 

The Baseline and Goal Data— The baseline and 
goal data should bje reviewed on each report. 

If no changfts have been made in this data, the 
report is satisfactory. 



If changes have been made in this data, the 
subgrantee should b* asked why they were 
made. Changes in this data arc only to be 
allowed^if the subgrantee had previously made 
a mistake or if he has more accurate informa- 
tion. 

The Matrix — for the data to be considered correct 
the numbers in liije 1 should be greater than or 
equal to the numbers in line 2, on a column by 
column basis. This same relationship should hold 
for lines 1 and 3. 



Note: Since wc arc measuring a flow at certain 
points^ the relationships described above may vary 
slightly from quarter to quarter, however, it should 
prevail over the period of the project. 

2. Baseline and Goal Analysis 

This analysis needs to be completed only when 
the baseline and goal report is submitted. 

If no legal aid is being given then only lines 1 and 
6 can be filled in. If some aid is being given then 
lines 2 and 4 can be filled in as well. 

3. Quarterly Analysis 

This analysis should be performed after the data 
has been found to be complete and accurate. 

. Compare lines 1 and 3 to access the scope of the 
project, „ 

Compare lines 2 and 3 to measure the effect of * 



the project and determine: 

whether the goals are realistic in terms of case 
volume and available legal manpower. Use 
baseline data as a guide. 

* If the goals are realistic, whether the ACTUAL 
aid given approaches the GOAL. • 

Compare lines 4 and 5, again measuring the effect 
of the project and determine: 

whether the goalj arc realistic in terms of avail- 
able manpower {1 man month 22 man days). 

1^ whether the available legal manpower is ac- 
tually being utilized, and 

whether the hours or days of counsel per case 
(line*2 : line 4) is 

— <:lQse to the GOAL (line 3 : line 5) 
-4 — sufiBcient to cause some benefit. 
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REGIONAL PLANNING UNIT QUARTERLY REPORT 
Corrections Quarter HI July-September 1973 

C-5 — Community Treatment for Delinquent Youth " * 
O00O-pO-C5-72 Family Center Diversion Project ^ 

Famity Center Diversion Project — 1972 , 



nSCAL STATUS 


1972 Grant Award 


$69,986 


Expended to Date 


69,986 


Balance to be Expended 




Start Date 


10/2/72 


Tentative Termination Date 


11/1/73 



The FamUy Center is geared to the pre-delinquent 
youth and his family. Early, intensive crisis inter- 
vention and short-term counseling is available six 
days a week. Staff members of the Family Center 
may* spend up to two or three hours during the first 
crucial points of intervention with the children and 
the families, with the overriding goal being to in- 
crease the capability of the family to function as a 
unit in coping with its problems. Diverting the child 
from the court system is a major objective. 

Last qukrter, 109 cases were referred ,to the Cen- 
ter, with a record of 44 cases in September. A goal 
of 50 per month was established as the maximum 



amount of families that can be effectively coun- 
seled with two full-time and two part-time counselors 
(the supervisor carries a half-load). The figure is 
subject to re-evaluation. 

The majority of referrals *to this project were 
made through Juvenile Court last Quarter (62% )• 
The remaining cases were mainly "walk-ins". The 
male/female ratio was about 50/50 with the major- 
ity of youth between the ages bf 15 and 17. Nearly 
half of the cases were in ninth or tenth grade, and 
over 60 percent were only first-time offenders, which 
is within the guidelines of the program, that of di- 
verting youth from the formal system. 
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OHI0-ExhiUt4 



Project No.. 



.Continuation D .SUBGRANTEE. 



EVALUATION INSTRUMENTS. 



RATING 



Quarter 
Ql 

Q2 
Q3 
Q4 



Ovtrall 



□ □ □ 

□ □ □ 

□ □ □ 

□ □ □ 



PROJECT EVALUATION SUMMARY 



PROBLEMS 



□ □□ 

□ □ □ 

□ D □ 

□ □ □ 

□ □"□ 

••^ Requires Commtnt 



Ql Comments: 



Actions: 



I I Visit j I Phone I [ 



Other 



Q2 Comments: 



Actions* 



^ I j Vtsit I I Phone | { Other 



Q3 Comments: 



Actions: 



Phone 



Other 



Q4 Comments: 



Actions: 



I I Visit j I Phone I I Other 



Comments on Completed Project: 



I I Recommended Continuation 
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.PROJECT EVALUATION SUMMARY 



Instructions for cwnplcting-^ 
U Project Number— The identification and control 
number assigned to each project. 

2. Continuation-^-Check the box if this project is a 
continuation of a prior project. 

3. Subgrantce—lnsert the short tiUe name of the 
fubfrantee* ' 

4. Evaluation Instruments — Use the numbers/let- 
ters identifymg each type of evaluation instru- 
ment which the subgrantec is required to com- 
plete and submit* 

5. Project Evaluation Summary 

Rating— Quarterly and at the completion of 
the project check one of the three categories: 
Outstanding— Check when the actual .pro- 
gress of the project exceeds its goals by 
more than 10% AND the goals are reason- 
ably difficult to obtain. 
Satisfactory— Check when the actual pro- 
gress of the project is from 10% above the 
goals to 20% below AND the goals are 
realistic. 

Unsatisfactory-^hcck when the actual pro- 
gress averages more than 20% below the 
project goals or more than 30% below any 
single goal, AND the goals arc realistic. 
NOTE— Where the goals are not realistic in the 
light of baseline data and other experience, 
they should be adjusted to realistic levels of 
performance BEFORE the comparisons to 
actual data are made for evaluation pur- 
poses. 

Problems— Outrterly and at the completion 
of the project check the applicable problems 
causing or substantially contributing to the 
unsatisfactory evaluation of the project. 
Unrealistic Goals — Check when the goals 
or anticipated accomplishments of the pro* 



ject are impossible of attainment by the 
project with the resources and structure 
available. If slight modification of the goals 
can be accomplished without chanpng the 
nature of the project then it should be done 
and noted on the evaluation instruments. 
Project Structure— Check when the pro- 
gress of the project is less than anticipated 
and the cause appears to be Attributable to 
the project management, organization, lo- 
cation or procedures. 

Project ppcration--Check when the pro- 
gress of the project is less than anticipated, 
the goals AIU3 realistic, and the project 
structure appear adequate. 

6. Comments— Quarterly and upon project com- 
pletion, write a few brief comments on high- 
lights and problems of the project. Comments^ 
are required whenever a rating of outstanding 
or unsatisfactory is given. Likewise the action 
taken following an unsatisfactory evaluation 
must be checked and commented upon. 

7. Summary Questionnaire— At the conclusion of 
the project provide brief answers to the five 
questions presented. These answers will help in 
reviewing the project at a later date for com- 
pilation of the Past Progress Section for regional 
and State Plans as well as, providing guidance 
for future planning. 

Answer briefly at the completion of the project: 

1. What was the projjtct goal? 

2. What were the projects accomplishments? 

3. What were the major problems? 

4. What were some of the solutions to those prob- 
lems? 

5. Comment on the Grantee answers to the narra- 
tive questions in the Evaluation Instrumcnt(s)! 
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OHia--EihibH 5 

PROJECT MONITORING REPORT 

^ Project Number: ^ Amount Awarded: ^ 

Project Title: 

Subfrantee 

Implementinf Afency: = 

Grant Award Date: 

Date Project Implemented: . 

Funds Encumbered : AJD 
Comments: 



— — = — — — -Telephone No, 

Termination Date: 



State Local 



Programmatic: A negative answer to any of the foUowing questions requires an explanation in the commento. 

A , 1. . Yes No 

A. Is the project proceeding toward iu stated objectives? 

B« Is Uie project utilizing the methods and procedures enunierated ^ 

in its application to achieve its goals? 

C Is the quality of staffing in conformity with the application? . 

D. Is the ImpIementin|rAgency keeping adequate records 

of its activities? 

E« If staff training is involvedjs it in conformity witii tile 

application in its quality and quantity? 

R If tile project calls for a special condition to be fulfilled, 

has it bpcn or is it being accomplished? ^ 

G. Are report requirements bemg complied wiUi and 

are reports being forwarded to AJD? 

CcmmifUs or Explanations: 



/ 
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finmcid: 

A. It the Implementing Afency conducting its aflfau^ 

conformity with iu budget? 
B- If the Implementing Agency drawing fundi it a proper rite? , 
C Are Mleqiutefinandal records beingnMuntiined? 
D. If Form 102 being used ind is it being completed properly? 

Comments or Explanations: 

Bvaluaiion: 

A. If the project hu been selected for tneviluition, is the 
Implementmg Agency maintainmg records on a continuing 
basis for this purpose? 

B. If specific methods for the collection of data have been esUblished, 
is the Implementing Agency conforming to the requirements 
established? 

Communis or Explanations: 



Yes No 



Yes No 



4. A SytNm That MMttors ActlvlHes AgrMd 
!• in Mvmntm ^ Ftnidlni 

South Carolina employs a two-part monitoring 
system, consisting of a quarterly financial and pro- 
grcM report completed by project directors and 
periodic site visits by RPU or SPA staff. The system 
actually begins during a pre-planning phase before 
applications are submitted. At that time^the SPA 
iuues a set of project criteria for each project type 
within each program area elipble for fundinjj*^ The 
criteria Identify the purpose of the project type, 
eligible agencies and general requirements, which 
include the variables to be used in the measurable 
objective statement of the subgrant application. 
Exhibit 1 gives the criteria for one such project 
type. (Criteria can be developed for projects not 
included in the master set but in which a potential 
applicant is interested). 

In completing the subgrant application, all appli- 
cants must provide a work schedule of necessary 
tasks for implementation, the person responsible, 
and expected dates of implementation and com- 
pletion. A timetable chart for this is provided in 
the subgrant application and is given in Exhibit 2. 

The Evaluation Section of the application has two 
parts. The first asks for a measurable objective 
statement using the variables listed in the project 
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criteria and a description of how the accomplish- 
ment of the objectives will be verified. Completion 
of the second part is required for those projecU 
which have internal evaluation as one of the project 
critarii general requirements. The applicant is asked 
to specify project activities and the quarterly planned 
level of accomplishment for each. Those projects 
completing the second part of the evaluation section 
are known as Type II projecU and each quarter they 
report on the actual achievement for each activity. 
Exhibit 3 gives the Evaluation section from the 
application, and Exhibit 4 is an example of how 
one project completed this section except for the 
quarterly projections for the activities. If the SPA 
Division of Planning, Coordination and Evaluation 
docs not consider the measurable objective state- 
ment or the list of project activities adequate when 
the application is submitted it will be returned to 
the applicant or held at the SPA until the deficiencies 
have been corrected. Generally, the SPA staff will 
hold a meeting with the potential subgrantce and 
regional planner to develop an acceptable application. 

When a grant is awarded, the project director 
receives a package containing the required Quarterly 
Progress Reports. M it is a Type II project, an 
activity page is included in each, on which the* 
activities and projected levels of achievement have 
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been copied from the application- Each quarter he 
filb in the actual level and sends the report to the 
SPA* Bxhibit 5 is a copy of the Quarterly Progress 
Report for a Type II pro|ect* 

The second part of the monitoring system, the 
on-site visit, is scheduled according to grant size* 
Projects with l^gc grants arc visited more frequently 
than those with smaller/ones* The responsibility for 
making the visits is ^ared by the SPA and the 
RPUs.^Thc SPA staff ipnerally visit larger projects 



and RPU staff smaller ones, but occasionally they 
visit together* Exhibit 6 is a copy of the Site Moni- 
toring Report and the instructions for completing it. 
After each site visit a feedback report is sent to the 
project director with a summary of the jBndinp by 
the monitor, recommendations and an overall rating 
of the project. If an unsatisfactory* rating is given, 
an accompanying letter notifies the project director 
that another site visit will be made after thirty di^$* 
to check progress made during that period. A copy 
of this rejwrt is given in Exhih^ ?• 



E Functional 
Category: . 
05 Program Area: 
01 Project Type: 
Purpose: 

Eligibility 



General 
Requirements: 



SOUIB CAROUNA—ExUbtt 

SAMPLE PROJECT FROM PROJECT 



'judiciary 
Judi(|^I Programs 
Public Defense 

To provide for the just defense of 
indigents in criminal cases* 

The Public Defender Association 
4ind Public Defender Corpora- 
tions are eligible. 

—Applicant mufst provide OGJP 
with a measurable objective 
statement and statement of 
methodology for measuring 
progress toward that objective, 

— The following variables will be 



CRITERIA 



EO501 



used in measurable objective 
statement: 

1) Qient selection criteria 
*( type of client) 

2) Expected caseload 

3) Reduction of conviction 
rate to be achieved 

4) Courts to be involved 

5) Geographical area con- 
cerned 

6) Time span of project 
—Personnel, equipment, supplies 

and other operating expenses 
directly related to the project 
are potentially eligible costs. 
—Internal evaluation required. 
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11. PROJECT DESCRIPTION 



SOtlTH GAROUNA--Exhiblt2 



TIMETABLE 

Provide a work schedule of what will be done at eaqh stage of the project ami estimated tlmAriter^^ tor 
each. This is an itcmtzation of the tasks and activitiesof tfte project who will handle each, and the initiation 
and completion dates of each. 
Examples 



TASK 



PERSON 
RESPONSIBLE 



EXPECTED DATE OF 
IMPLEMENTATION 



EXPECTED DATE OF 
COMPLETION 



Interview 

Hire 

Train 

Assign duty 
Dally count 



Cty. Superintendent 
County Board 
Dept. of Corrections 
Cty. Superintendent 
Jailer 



Jan. 1. 1974 

Feb: 1, 1974 
Mar. 1, 1974 
Jan. 1.J974 



Jan. 15, 1974 
Jan. 15. 1974 
Mar* 1. ft74 
Ongoing 
Dec. 21. 1974 



TASK 


PcRSON 

RESPONSIBLE | 


FX:PErTED DATE OF 
IMPLEMENTATION 


EXPECTED DATE OF 
COMPLETION 








• 




*■ 


> 




% 




1 ^ 


4 



B, NARRATIVE ^ 

Further describe the proposed project on attached sheets. Elaborate as to other situations and outcomes 
J^^imem^mlmi as precisely as possible how the project will be carried out D.«uss 
all methods and activities the project will incorporate. Discuss »ll/equir^men.a other than the measur- 
Jble statement and internal evaluation' requirement listed under "General Requirements" of the project 
criteria. Attach pages as necessary. 



♦Denotes Change 



SOUTH CAJtOLlNA-Exhibit 3 ^ • 
^ Evaluatioa Scctkm 

v . - 

•IV. EVALUATION \ 

A. PROJECT RESULTS 

1. MEASURABLE OBJECTIVE STATEMENT 

|;'«'^«'*« items listed in the project criteria under J-General Requirements" (the phrases with numlwrs 
beside them), write a sentence(s) that tells exactlik what will be the situation once the purpose of the 
project has been achieved. . " r 
Example: "Tm jailers will be hired within three months thereby achieving ratio of four con*ctional of- 
ficers to eighteen innwtes." 



Z VERIFICATION 

List 1) what documents and measurements will be kept, and 2) by whom to verify the accomplishment of 
the measurable objective statement. 

Example: "County Manager ~ records of officers' employment and data Correctional Officers — dally 
count of inmates." 

LOOK AT "GENERAL REQUIREMENTS" OF THE PROJECT CRITERIA. IF INTERNAL EVALUA- 
TION IS NOT REQUIRED, STOP. THIS GRANT APPLICATION IS COMPLETED. IF INTERNAL 
. EVALUATION IS REQUIRED. COMPLETE THE REST OF THIS APPLICATION (WITH THE EX- 

^ AMPLE USED. THERE IS NO INTERNAL EVALUATION REQUIRED. IF THIS SECTION RE- 
QUIRES COMPLETION. YOUR PROJECT WILL BE CUSSIFIED AS TYPE II FOR PROGRESS 
REPORTS.) 



*0«notM Change 
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1. What data about th*? project's activities will be collected to measure the efficiency and petfc i: ancc of 
this project. How tiiuih of each activity do you pnedict oct jrring. 

Example: j*-'-^''^ 



Activity 


Totaled AnK)unt t^J^^Months 


Hours counseled 
Mo* of clients 
No. of recidivists 
Mrs, of equipment use 
No. of felony offend- 
ers reported in cty. 
No. of fonow-up 
clients 


3 mos* 


6 mos. 


9 mos* 


24 
4 

618 


76 

17 * 
1439 


142. 
60 
3 

2019 
3 



12 mo% 



180 
GO 
3 
28 

1732 

16 



Activity 



Totaled Amount by Months 



3 nios. 



1 6 mos. ! 



9 moo 



12 mo3 



Sl^;^i?SfdS-mif.e .{'tiie project i5 suceersful (uoide u,m S.cti&.i II d C-^"' f ^^'^^ "f'^*! 
is yo!r idel) am-) fesired ou.rornuCui Write in numerical tem,3 .Amv Kosoible., Attach odd.t.^-.l 

tomplj ""Inmft'S will b^s alkct-d 8 rnste houfe per weel of recreation and/or visitGlie.i," 
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EXAMPLE OF COMPLETED EVALUATION SECTION 



75-002 



Measurable Objective: 

Seven hundred fifty indigent clients (those who so 
plead and are selected on basis of a background 
check) wDl be served by an attorney at a client- 
attorney ratio of ?23:L These clients will be 
heard in the Court of General Sessions and other 
courts ot Charleston County. A 1 percent reduc- 
tion in the current rate of conviction of indigent 
dient^ in the Court of General Ses^ons will be 
achieved within 12 months. 

Verification: * . 

Summarized court records reflecting the current 
rate of conviction of indigent persons heard in the 



Charleston Court of General Sessions: 
Summarized court records reflecting the rate of 
conviSlion of mdigent persons heard in the 
Charleston Court of General Sessions after project 
has been operational 12 months; 
Total number of clients represented by Charleston 
County Public Defender Corporation; 
Copy of background report for each client se- 
lected by the Charleston County Public Defender 
Corporation. 



Definition: Cgnviction — Only those *^persons ^ho 
plead not guilty and are then found 
guilty and sentenced. 
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Activity 



Project Frequency/Amount 

3 mos. 6 mos. 9 mos. 12 mos. 



Contact potential clients 

Verify indigency 

: Accept clients 

Assist in obtaining bond 

File motions 
Type 

a. Pretrial 

b. Trial « 

c. Posttrial 
Outcome 

a. Accepted 

b. Denied 

No. of cases plea*bargained 
Trial 

No. of cases brought to trial 

Average length of trial by week, V4 day, and day 

Outcome of trial 

a. No. guilty 

b. No. not guilty 

Investigations 

No. of clients investigated 
Type of investigation included: 

a. Personal background 

b. Employment background ^ 

c. Criminal background 

Average length of investigation by week arid month 
Frequency of investigation at 

a. Pretrial 

b. Presentence 

c. Posttrial ^ * 
No. of cases continued to contingency pocket 

Appeals 
No. of appeals 

Speci^^ juri!^diction and number ^ 
a* ^ , 

c. 

No. if indigent clients represented for repeated 
offenses 



SOUTH CAROLINA— ExhiUt 5 

GOVERNOR'S COMMITTEE ON CRIMINAL JUSTICE 
CRIME AND DELINQUENCY 



ACTION GRANT PROGRESS REPORT 
-1. DistAct name and code:' 

^ ■< — — 


5. Grant Number* 

6. Date of Last Report: . 




to 




7. Grant Amount! V 
o (Federal) 

8. Period Covered by Report: 


2. Implementinf Subfrantee: 








3. Title or Character of Project: 
f 


GRANT INFORMATION 
9. A. Date 53uhmitteH 




B. DateAwirHeH 




C. Date of Last Fund Reauest / 

D. Date of Last Fund Receint 

E. Total Funds Receiver! 


4. Matching Contributions Received: 
A. CMh 


B. Buy-Tn 


10. Date Proiect Imnlemented! 


If not received, please explain • 


If not. please explain ! 










A. Interim 




B Fifi«I 




Type Name and Title 6f Protect Director 


Project Director (Sigmttwe) 
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GOVERNOR'S COMMTITEE ON C3UMINAL JUSTICE 
CRIME AND DELINQUENCY 

Quarterly Progreii Report— Type II 



Grant #- 



Yes 'iVo N/A 



1, it the project on schedule? ^ — — 

2, Ii staifing * * . 

t) on schedule? • 

b) mectmg job description requirements? > 

Attach a resume of each new project employee hired during this project. 

3, Is facility construction, renovation, or acquisition 

a) on schedule as outUned in the Timetable of the grant application? ^ — 

b) meeting project specifications? , 

4, Has the equipment been 

a) ordered? 

b) paid for? ' . 

c) put into use? — — 

5, Has this project received any written publicity or community response? 

6- Has this proj^t operated without problems during this quarter? ^ 

If negative answer, please respond with an attached narrative answeruig 
the questions 1 ) what problcm(s) arose, 2) was corrective action 
necessary,^ ) was it taken, and 4) what was the corrective action? 

7. Will the current rate of expenditure allow for full use of project funds 
by the expiration date of the subgrant period? 

8. Hm an on-site monitoring visit been made duruig this quarter? ^ 

9. Are special conditions being complied with? ^ 

10. Attached are this project's activities and the projected total amount of each activity through this tunc 
period (as outlined on page 12 of the grait application). Would you supply the actual amount for each 
activity from project beginning to date. 
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S0U1H CAROLINA— EsUUt € 

GOVERNOR'S COMMITTEE ON CRIMINAL JUSTICE, 
CRIME AND DELINQUENCY 

MONITORING REPORT INSTHUCTIONS 

The following information is to be submitted on the OCJP n>onitorin| report form. Completed reports 
should be mailed to the Supervisor of Coordination within five (5) days after monitorinf viiit. If there arc 
any questions, call the Supervisor of Coordination for assistance. 

Question 1— IMPLEMENTING AGENCY * ■ 

Refer to Block 9— Grant Application 

Question 2— AGENCY DIRECTOR 

Refer to Block 3 and Block 9 — Grant Application 

Question 3— PROJECT TITLE 

Refer to Block 5 — Grant Application 

Question 4 — Grant%mount-^ce Block 7— Grant Application * ^ 

Grant Period — See Block 2 — Grant Application ^ 
Continuation---Sec Block 3— Grant Application 

Question 5— PROGRAM OBJECTIVES * 
Refer to Page 1 1 , Section IV, Paragraph A, Part 1 of Grant Application 

Questien 6-^SUMMARY STATEMENT 

To be completed after monitoring visit is conducted. This should be an objective and subjective analytii 

, of the monitoring visit findings. . 

Question 7— PERSONS INTERVIEWED 
Self-explanatory 

Question 8~SPECIAL RECOGNITION 

Refers to awards and citations, not news coverage 

Question 9— STATUS OF PROJECT 

Refer to Page 1 0 of Grant" Application 

Question 10— IMPLEMENTATION 

Refer to Page 10 of Grant Application 

Question 1 1— SPECIAL CONDITION COMPLIANCE 
Refer to Grant Award 

Question 12— DATA COLLECTION 

Refer to Page 1 2 of Grant Application ^ 

Question 1 3— PUBLIC OR COMMUNITY INTEREST 
Refers to News Coverage 

Question 14— FUTURE iFUNDING 
Self-explanatory 

Question 15— EXTERNAL RESOURCES 

Should include all agencies, financial support, donations, etc. utilized by <hc project tv 

Question 16— CONSULTANTS 
Sclf-explafnatory 
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Question 17— PROJECT TYPES 
Indicate appropriate type 

Question 18— ATTACHMENTS 
Indicate those attached 

ATTACHMENT A: TRAINING 

1. IN-SERVICE TRAINING— self-explanatory 

2. TRAINING FREQUENCY— self-explanatory 

3. OTHER TRAINING— list type, frequency, number attending 

4. STAFFAPPRAISAL— This should be the opinion of a participant in the training. 

5. TRAINING EVALUATION— self-explanatory 

ATTACHMENT B: PERSONNEL 

1. EMPLOYMENT— self-explanatory ^ 

2. QUALIFICATIONS— refer to job descriptions which were attached to grant application 

3. ' PROBLEMS— self-explanatory " 

4. TURNOVER— self-explanatory 

5. DISCRIMINATION— self-explanatory 

ATTACHMENT C: EQUIPMENT 

1. EQUIPMENT PURCHASES— refer to Page 7 of grant application 

2. EQUIPMENT SPECIFICATION— refer to Pages 10 & 1 1 of grant application 

3. EQUIPMENT PERFORMANCE— self-explanatory 

4. EQUIPMENT/PERSONNEL— self-explanatory 

, ATTACHMENT D: FACILITY 

1-10 — self-explanatory 

11. EXPLANATION OF POOR AND INADEQUATE FINDINGS— If any block is checRed-wEichdtes 
' negative conditions, these should be documented. Documentation would incliide both the item number 
iand the reason for cheeking "poor" or "inadequate". 

Example: 

Item 2. There are no bus lines; street signs are not visible. 

. ATTACHMENT E: EVALUATION ^ 

1 . DATA COLLECTION— refer to Page 11 and 1 2 of grant application 

2. EVALUATION PERFORM.WCE— refer to Page 1 1 and 12 of grant application 
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ATTACHMENT F: SERVICES ^ 

1, SERVICES PROVTOED— self-explanatory 

2. PROJECTED SERVICES— refer to Page 1 1 and 12 of grant application 

3. PROJECTED POPULATION— refer to Page U and 1 2 of grant application 

4, REQPIENTS OPINION — Check only the written or oral opinions of recipients, not project personnel's 
■ opinion of what the recipients think. 
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SOUTH CAROLINA OFHCE OF CRIMINAL JUSTICE PROGRAMS 
SITE MONITORING REPORT 

Monitor Name: — - Date:_ _ Grant No:„ ^ 

1. Implementing Agency — — - — — — ^ - ; — - — — ■ - — 

Project Director— ^ ^ ^ — - — — 

Address — ^ — — — — ■ - — ~ — — — — : — — =^ 

2. Agency Director or Financial OflScer — — ^ — — — v-- - — — — — — • - ■ 

Address^ — ~^ . - - — — ^ — — ~ — 

3. Project Title..^ - — — — — 

4. Grant Amount: TOTAL - — Federal $ State $ Local $ 

Grant Period (Award Date)^ to_ new. _ continued — — 

5. Specify Program Objectives: 

6. Summary Statement: (important stages, accomplishments, problems, etc, to describe current status of 
project) 

7. Numlber of persons interviewed during.monitoring process — — Name and title of person interviewed. 
1. 

2. 

3. 

■4. - ' ' ' , - ■ • 

5. . .* ■ 
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SOUTH CAROLINA OF CRIMINAL JUSTICE PROGRAMS 
SITE MONITORING REPORT 

8. Special Recognition: 

9. Status of Project: ( ) No Progress ( ) Initial Stages ( ) Fully Operational 
( ) Nearly Complete ( ) Completed 

10. In comparison to Implementation Schedule, Project is: ( ) On Schedule ( ) Ahead of Schedule 
( ) Behind Schedule * ' 

1 1. Subgrantee complying with special conijitlons of award? X ) Yes ( ) No ( ) N/A 

il. Is data collection for evaluation (or otherwise) a component of the project? { ) Yes ( ) No 
( ) N/A 

13* Evidence of public or community interest in project? ( ) Yes — favorable ( ) Unfavorable 
( ) no evidence of interest or involvement 



CITE: 

I. 

14. How will project be funded at termination of Federal assistance? 

( ) Known^ — Cite: ( ) Not known— Cite possible:. 

15. Has the project utilized external resources? ( ) Yes. List: ( )No 



16. Have consultants (individual or group) been employed? ( ) Yes ( ) No 
If yeSt attach copy of contract or fmal work product* 

17. Does this project deal primarily with (may be more than one) ^ 

( ) Facility ( ) Services ( ) Training 

( ) Training ( ) Equipment ( ) Pejsonnel 

^ ( ) Other, List: 



18. Mark which attachmenjts follow (jm^iy be more than one) . 

( ) A Training ( ) B Personnel ( ) C Equipment 

( ) D Facility ( )"E Evaluation ( ) F Service ' 



; SITE MONITORING AITACHMENTS 

A. TRAINING 

L Are personnel receiving in-service traininfe? ( ) Yes { ) No 
82 V 



. M yes, is there a curriculum or written record of course content? 
( ) Yes—Attach ( ) No— Briefly describe 

0 



2. It triining received on regular basis? ( ) Yes ( ) No 

3. Briefly describe all other training. > 4 



4. Std! appraisal: ( ) Good ( )Fair ( ) Poor ^ 
5- Arc you evaluating the effects ot training? ( ) Yes— Describe ( ) No 



PERSONNEL 

1, Arc jjtt personnel hired? ( )Yes ( )No 
iX) employees possess qualifica^s as stated on job description? ( ) Yes ( ) No— Exploit 



3- Problems mentioned by staff? ( ) Yes— List "{ )No 



4. Is there a high turnover rate among project personnel? ( ) Yes ( ) No ^ 
$«J[s there evidence of discrimination m personnel as to race, sex, religion, or national okgtn? 
^ T>««^itc ( ) No 



C EQUIPMENT 

1 . What equippient has been purchased (may be more than one? ( ) General ( ) Office 
( ) TrandSportation ( ) Personal ( ) Radio ( ) Other— Cite 



2. Does equipment meet specifications stated in grant application?. ( ) Yes ( ) No — explain 



3. Docs equipment meet expected performance standard? ( ) Yes ( ) No— explain 



4. Is properly trained personnel available to operate equipment? ( %) Yes ( ) No ( ) N/A 
D- FACILITY 



) Good 
) Good 
) Good 
) Good 



EXTERIOR: 
L General Appe^ancc ( 

2. Accessibility ( 

3. 'Grounds ( 
4* Parking ( 

INTERIOR: 

5. General Appearance ( ) Good ( 

6. Space: ^ ( ) Excessive 

7. Furnishings ( ) Adequate 

8. Lighting ( ) Adequate 

9. Climate Control ( ) Adequate 
10* Functional ( ) Yes ( 



( ) Fair ( ) Poor 

( )Fair ( )Pdor 

( )Fair ( )Poor 

( )Fair ( ) Poor 

( ) Clean ( 
) Inadequate 



) Fair { ) Poor 
( ) Adequate ( 
( ) Inadequate 
( ) Inadequate 
( ) Inadequate 
)No 



) Soiled 



1 1 . Explain all poor or inadequate ratings (indicate number). 

E. EVALUATION 

L If data is being collected for evaluation purposes* check which ^pes of data: 
( ) Reported Crime by ( ) State, ( ) County, ( ) Department 
( ) Conviction by ( ) State, ( ) County, ( ) Department 

( ) Arrest by ( ) State, ( ) County, ( ) Department 
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( ) Response Time 
( ) Patrol coverage 

( ) Rate of recidivism by ( ) State, ( ) County, ( ) Department 
( ) Financial 

( ) Personnel (staff) ^ ? 

( ) Use of equipment 

( ) Change in procedures by ( ) State, ( ) County, ( ) Department 

( )aient ^ 

by ( ) age ( ) sex ( ) race ( ) offense ( ) service reived 

( ) socio-econwnic status ( ) education ( ) employment ( ) recidivism 

( ) Opinion of project ( ) management's ( ) Staff's ( ) Clients' ( ) Other Specify. 



( ) Other data; specify. 



2. If evaluation being performed as outlined in grant application? ( ) Yes ( )No — Comment 
SERVIGB^ 

1 . What service (i ) docs program provide? 

Number of Recipients 

(a) _ — 

(b) — 

(c) '- ^ • — 



2. Are these in agreement as to number and type with projected services stated in grant application? 
( )Yes ( )No — explain 



3. Are these ntmibers in agreement with project population of grant application? 
( )Yes ( ) No— explain 



4. How do recipients view the project? ( ) EflEective ( ) Ineffective— Cite ( ) No opinion 
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SOUTH CAROLINA— ExUUt 7 

SOUTH CAROLINA OFHCE OF CRIMINAL JUSTICE PROGRAMS 
MONITORING FEEDBACK REPORT 

Project No.^ — Date of Award Date of Last Monitoring^ 

Title of Project: - ^ , - — . - - ^ - „ 

Monitoring Perind: ^ / tn 



Grant Amount: TOTAL , FederaL State. . LocaL 

Purpose of the Grant: 



Scope of Review - 

This monitoring was conducted to determine whether or not the procedures set forth by the grantee in 
this grant application arc being maintained as stated. The visit was conducted with generally*accepted moni* 
toring standards, and accordingly included inspection of the project and other monitoring procedures as was 

necessary in the circumstances. 

Summary of Findings* " ^ 



Recommendation: 



CHECK ONE: Satisfactory .Unsatisfactory 
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SACRAMENTO, CALIFORNIA -Exhibil 3 




=1 CALIFORNIA COUNCIL ON CRIMINAL JUSTICE 



7171 Bowling Drive 
P.O. Box 9532 
Sacramento. California 95823 



QUARTERLY PROGRESS REPORT 



1. PROPONENT (name and address) 



PROJECT DIRECTOR: 



2. TYPE OF REPORT * 

DUE SUBMITTED 
^ 1st Quarter t ] 

2nd Quarter t ] 

3rd Quarter [ ] ^ 

4th Quarter [ ] 



3. PROJECT TITLE 



CCCJ ^. 
REGION. 



4 REPORT PERIOD. 

REPORT DATE 

PREPARED 8Y. 



TITLE. 



/ 



CCCJ #508 6/73 
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INSTRUCTIONS FOR^MPLETING QUARTERLY PROGRESS REPORT 



" IFbrward two copies of this Report to the appro- 
priate Regional Criminal Justice Planning Board 
fttff. 

* The Regional Planning Board $Vdft will be respon- 
siT>le-for-providinrat-least^"Orte-pa g e-evaluation 



of each progress report. A copy of tbcv progress 
nd-thc-tnaIy$i$^iIH>c^ forwarded to: 

Planning Operations Division 
California Council on Criminal Justice 
7171 Bowling Drive 
Sacramento, California 95823 

1. Proponent: Enter the name and address of the 
proponent. Include the project director with his 
contact phone number. 

2. Check appropriate box. 
3* Self explanatory. 

4. Reirort Period: Indicate dat^s included in* the 
quarter. 

" Report Date: Date of completion of the report. 
Prepared by: Indicate name and title (relation- 
ship to project) 

5. Narrative: Describe briefly the implementation 
of the project during the quarter* Include com- 
ments on the following subject areas. You are 
requested to address each heading. 

I. SUMMARY OF ACTIVITIES FOfe 
PERIOD 

A/ Activities and Achievements 

1. Briefly discuss the objectives to be met 
and status of achieving the objectives 
for this report period. Inclddc $ de- 
tailed analysis of prqject progress with 
respect to the work schedule in the 
grant contract. 

2. Provide a detailed listing of agencies 
utilizing the ser^dees provided by your 
project and agel^ies whose services 
your project is utift^ing* If your pro- 
ject provides services to individual 
clients identify the specific sources of 
your clients (e.g. probation depart- 
ments, family service agencies etc.) I 

B Problems Encountered 

1. Discuss any problems— indicate any 
delay in task completion dates of two 
weeks or more. Indicate the expected 



impact on the total project completion 
date as the result of any delays that 
may have occurred. 

Addressing the following points: 
4a} Statf^ii^h^e~l^i^ po'tsi^ 



(b) 



filled?' If not which positions re- 
Inain vacant and"whyt 
Equipment ordered 



♦Specifications published 
♦Bidder List 
♦Manufacturer 

(c) .Equipment Received 

(d) Cumulative grant funds awarded 
and cumulative grant funds ex- 
pended. 

IL ANTICIPATED ^ACHIEVEMENT S— v 
NEXT QUARTER 

A. Discuss significant achievements to be 
accomplished during the nexWeporting 

period* 

B. Thoroughly discuss any anticipated de- 
lays* Indicate the potential influence on 
the overall project time schedule. 

D. Exampbs of T«chniquts Ustd to Focus 
Monitoring Reports on Stitctod Issuts 

Two examples of techniques used to focus moni- 
toring reports on specific issues arc g\\cn here. The 
^st example illustrates on? of the more common 
approaches— using a standard reporting form that 
identifies topics to be discussed in narrative form. 
The second example illustrates an approach that is 
not as common as the first. In the second example, 
the monitors are required to review projects on 
specified performance factors and then reduce their 
assessment to a pass or fail relative to selected 
categories of the indicators* 

1* ExampUt of Sfandardiztd Narratlvt Re- 
porting Form 

The Alameda County, California, kegijbnal Plan- 
ning Unit uses a more derailed, txp^v^ version of 
the evaluation^ report required by the state.^ The 
RPU monitor is required to make a site visit to the 
project before completing the report. On-site Instruc- 
tions have been developed to guide the monitor artd 
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ensure comprehensive coverage of tU arets« On-site 
Instnictions^ in Exhibit 1, set forth the mi^itor's 
responsibilities, give guidance in preparing W and 
conducting the site visit and provide a distribution 
list for the report. The Interim Evaluation Report 
format in Exhibit 2 is completed after the site visit. 



surances of compliance with the grant award require- 
ments and fiscal information. 

^The California SPA recently chanicd its rcportinf 
requirements for the rcfions. The state evaluation report is 
required for each project only twice a year. In the fourth 
and eleventh months. Reporting is still required of the sub* 



^i:^gM s a h igh l )Latructti rr ri , r nm prff hcnti v e^eport^^^^tr«ntee-f6rHhe-^^ 
by specifying major areas to be addressed and listing i uidclines* However, the SPA is presently rcvisinf its ra- 
the itcms"to~bc covucdJoTqp^^ answered quirements to-jivc the rcftons^^mbrrff^^^ 
under each. Extensive information is sought on pro- subgrantces for these two quarters. All quarterly 
:-^f * ^ I . f • reports arc fcquirtJ to be forwarded to the SPA after 
jcci management ana aciiviiies in audition to as- veronal 
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ALAMEDA, CALIFORNIA—ExhiMt 1 



ON-SITE INSTRUCTIONS 

OFFICE OF 
XR1M1MAL^U1SUCUUINN11!I& 



-iUJU(UOA-lUGIQMALJCllMU!lAL.UlSlLa_Ui^ 



WEBSTER STREET • SUITE 104 •* OAKLAND, CALIFORNIA 94607 • 874-5274 
MEMORANDUM ~ April 30, 1974 

To, GRANTS MANAGEMENT AND EVALUATION UNIT 

From: PAULA a 
Sub[«€t: ON-SITE INSTRUCTIONS / 

For the purposes of these instructions, the term "on-site" will be used to refer to all contacts (both In 
person and over the phone) which occur relative to the requisite visits. Although these contacts constitute.* 
form of evaluation, for certain projects they. represent only a portion of the evaluation responsibility. The 
distinction has been pointed up for clarity only. The w4ten1io<au»lents will be termed "evaluation" reports. 

On-site Responsibilities \ 

t. To inquire into all aspects of project operations ahlvsk*^^^<^ related information for the purposes of 
determining compliance with contract, to identify problems which might affect the fulfillment of project 
objectives, and to document project status and progress. 

2. To provide or cause to be provided all technical assistance necessary and available within regional 
resources or to assist project staff In obtaining outside support in order to provide the project with every 
opportunity to achieve its stated objectives. 

3. To recommend corrective actions to inriprovc project performance or to remedy existing problems. 

4. To recommend grant revisions or amendments as necessary to assist the project to fulfill project objectives. 
Frequency of On-sites 

1. Staff must perform on-site visits during the 2nd, 5th, 8th. and 1 1th months of project operations. 

2. For projects of less than 12 months .duration, staff will adhere to the above schedule, as long as the project 
is in operation. ^ 

3. For Mojects of more than 12 months duration, visits will occur beyond the 1 1th month at intervals of every 
3 mon^s, as long as the project is in opcrittion. 

4. Each project will require an additional final visit if the last visit occurs prior to the termination of project 
operations. Final reports afe due within 90 days of the end of the project. 

On-site Approach 

L Preparation in volvck: ^ ^ 

a. Review of the contract, special conditions, contract amendments and regional rccomnicndations. 

b. Review of the latest 201» budget revisions and any audit reports completed on the projei:t. 
c/ Review of project quarterly reports. 

d. Review of all previous on-sitcs and special reports completed on the project, 
e* Review of pertinent correspondence. 
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Grants Management and Evaluation Unit 
Paula 

On-site Instructions 




April 30, 1974 



1 Review of prior years* contract files. 

g. Discussion with previous reviewer (s) ♦ if possiblcr^ 

h. HRcvtew^oHheT>roje?rrtilc i — — 



^CondueiingOn-sltcs " — " " ^ - -- — 

Schedule visits. Do not attempt to supervise project staff. / ^ * 
b,. During the prep^rsjtion stage, identify Ihisic contacts— e*g., project staff, sponsor rcprcscntitlves, policy- 
makers, user agencies, clients, -etc. It is anticipated that the persons and agencies initially identified will 
provide leads for other relevant contacts* ^ 



a Contacu will probably include visits as well as phone conversations* Very few, if any, on-sites can be 
adequately conducted on the basis of one visit* Dcpendini upon the natun| of the project and the 
reviewer's previous fifiiiliarity with it, do whatever is necesiiry for a thorough iinquiry. 
d. Use thejipprbach embodied in the attached on-site for every visit, regardless of the required format for 
-thcAvritten report. In addition to this basic approach, investigate concerns identified during the prepara- 
tion stage. It is anticipated that the basic format will serve only as a beginning, from which relevant 

points of inquiry will be pursued in greater detail. 

* ~ ' ■■ ^ 

3. Post-visits 

a. Di^uss findings with project representatives. Depending ^pon their feedback and concerns, staff may 
wish !6ix:larify written reports. The extent to which this octurs should be determined on a case-by-case 
basis. \ ^ - * 

b, Folldw ftiFough on all concerns identified via on-sites and on all actions initiated as a result of them. 

Written Reports ^ ^ ^ 

1. First Year of Project 

a. The 1st and 3rd reports (2nd and 8th months) will be presented in the abbreviated Version of the 
attached format. The primary difference lies in Section D (Project Operations), instructions are con- 
tained within the report. Consider these instructions as^ minimum requirements. The complete formal 
may be utilized if appropriate or if desired* 

b. The 2nd and 4th reports (5th and 11th months) will be complete versions of the attached format. All 
aspects of project operations that are appropriate to the nature of the project and have thus "far been 
documented should be discussed. Instructions are contained within the report. Consider them as mini- 
mum requirements, providing a basis for additional inquiry as relevant. It is anticipated that the 4th 
report will befar more comprehensive than the 2nd one* 

c. Final reports will vary considerably with the extent of staff's evaluation responsibilities. 

(1) Staff with minimal, if any, evaluation responsibilities will^uie the final report to include description 
of the year's operation (unless the last on-site coincides with the end of the project year, in which 
case a final report will not be required). In these instances, the final report may be nearly identical 

t to the last on-site, with very few revisions, e.g., an update of the cash flow section. If the project 
^ is a continuing one, staff may combine the findings relevant to a final report with the first on-site 
of the succeeding year. 

(2) Staff with major evaluation responsibilities will be required to complete a fintl report that is, in 
most instances, a far more comprehensive treatment of the prdjcct* The format for these reports 
will be developed on a case-by-case basis but will, as a minimum, contain the sai^ basic descrip- 
tive information a$ in the attached format. 



Although visitrand tlTe on-site approach will still be employed th£2nd^ 5th, 8th, and Uth months, the only 




Second and Third Years otProject 
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Grants Management and Evaluation Unit. 
Paula 

On-iite Instructions 



April 30, 1974 

reports which4tui bi required are those corresponding to the 5th and 11th months. (The previous year's 
final report will also be issued the third month of the new project year.) During the 2nd and 8th monOvh 
staff should subitiit jncmos to the file^ntaimng-oa^ it r f indings an d ^pportivc do0uittei)UtIo n^ 

Copies of the on-sites will be distributed u follo^^: 

• Project file 

• Assigned staff 

• Other professional staff 

• OOP 

• Official project director 

• Functional project director 

• Sponsoring agency representative(s) 

• Planning Board 

• Citizens specifically requesting material 



\ . 
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. ALAMIDA 
MGIONAL 
CMIMINAL JUSTICI 
HANNING ■CARD 



ALAMEDA, CALIFOIlNIA>-EiUbit 2 



OFFICE OF 



CRIMINAL JUSTICE PLANNING 

100 WEBSTER STREET • SUITE 104 OAKLAND, CALIFORNIA 94607 

Admlniitrotion (415U74^U 



' <? rt ^ nf^ > fa n crg» m t n t an d E vg lu ot } on'^4t5)-874-327 * 
RcMorch and Planning (415) 8Z4-7595 



INTERIM EVALUATION REPORT 




Grantee: • Project No.: 

Project Director: Grant Amount: 

Grant Period: Year Df Funding: 

Period covered by report: Report: 

Evaluator: Dateof Rejjort: 



This report shbuMbe cumulative, covering the period from the date of the project to the on-site date. 
A. EVALUATION SUMMARY AND RECOMMENDATIONS ^ ^ 

^ Highlight thp major findings of the report and provide specific recommendations for improving project 
performance and for correcting problem areas* 

The summary of the. report should include the following (include in final as appropriate) : 
1* State whether the project is in compliance with the grant, includirfg any special conditions stipulated 
by the Board* 

2. Identify the areas of significant progress made by the project* 

3. List any significant problems with the project identified during the period covered by the report* Be' 
specific in differentiating problems in the administration of the project, project personnel, delivery of 
services, etc* * 

4* Provide an assessm/»nt of overall project success or value/ 

Recommendations should Include consideration of the following?. 

i* State specific recommendations for improving the project's performance* Recommendations may 

identify areas for additional technical assistance* ^ 
2* For each identified problem area, provide specific corrective recon^j^ndations* 
3* Indicate whether there should be any amendments or revisions ,to ^e project or contract*^ Should 

special conditions be stipulated? 
4* If tHc report is being prepared prior to a coasideration for refunding! a recommendation for continu- 
ance br termination of OCJP funding should be provided. Is the project appropriate for criminal justice 
funds? 

5: Should an action to terminate the contract be initiated? If so, state in detail the feasons for this decision 
i and what thcjje actions should be* 

/ ' ; 4 . 
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The discussion of reconimendalions may be inlegraied within ihc major findings. ^ 
Use subheadings if appiopi iaie. The entire section should be set up to permit ready access to conclusions 
and recommendations- • 1 

. 1 • \ 

B, PROJECT SUMMARY 

Briefly summarize the major project elements in the lust paragK^nh. Include objectives and basic details 
about^ihe^i pproach The summ ary shuuld be concise yet completrVjUgh to give somconeunfamiliar with 
it a good grasp of what it is trying to do and how tt is going about it. " 



C PROJECT OPERA HONS ^ 

t First/rhird and^oufttHieports * , 

Describe progress to date, constraints on operations, problems encountered in mcctmg objectives, 
methods of resolving probleiiis, and current operational status. 

The organization of this seciion.and the level of detail will be determined by what needs to be said at 
this lime about the project. 

Although the fust, third and fourth reports will reflect only major findings, the approach utilized by staff 
should be just as thorough as for the second report. Consequently, the lines of inquiry listed below 
should be pursued each time. ^ 

2. Second and Final Reports ^ ' u u 

Describe project activities in full The organization of this section will depend somewhat upon the 
nature of the project but should generally include the major sections and subsections listed below. 
The format of this sectioH will undoubtedly require major modification or the addition of other major 
section headings in the final evaluation report. For example, Sn addition to a major section heading for 
project operations or activities, a section might be added for project impact on clients, the system, or 
the community or for a cost effectiveness analysis, etc. 

a. Organizational Structure ^ - 
( J ) Identify the poUcy-njaking body for the project. 

(2) Delineate lines of authority and supCf\isions within'the project. Discuss the role of advisory 
bodies, if any. Describe 'the formal and informal set-up. 

(3) If applicablj^scuss the relationship between the sponsoring agency and the private implc* 
menting i 

(4) Include a/ organizational chart. 
(5 ) Problen/ in of these areas should be fully developed or a statement made that their func- 
tioning t^ssjiKsfactory, 

b. Staffing 

(1) List in a c^unmSjll personnel ever employed by the project. List in adjoining columns their 
iitles, datcs\>f empkivment, and dates of termination. Unfilled positions, should be listed by 
title with the yord "unfilled" placed in lieu of the employee's name. 

(2) State whclherXhe staff positions listed above were authorized by the grant contract, why posi- 
tions remain tAifliled, or any particular reasons for staff termination. If turnover requires 
further explanation, fully develop this area and indicatc^the extent to which it has affected 
project operations. 

(3) Do the personnel currently employed by the project meet the job specification requirements 
outlined in the grant contract? Are the personnel salaries consistent mth the grant contract? 

(4) Describe the functions of project personnel Do these adhere lb those described in the contract? 
How do staff relate to each other? 

(5) If consultant services are included in the contract, what are the specific functions? Do they 
seem to be providing the quality of serv#i required? ^ 

(6) Identify any problem areas not already discussed, < 
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c. Staff Training 

(1) Are specific staff training needs ideijtified in the grant contract? If so, has (the training been 
* . provided? What other training has been provided or is planned? . * 

(2) Has the operational experience of the project indicated specific areas'of nec<t for specialized 
staff training? i ni 

(3) Do^this area present any problems? , * / ' ^ 

d. Project Philosophy , 

( 1 ) If appropriate, identify the orientation of the grant developers and, if different, the grant imple- 
mentors (st^), the cooperating agencies, and the clients. 

(2) Have there bean any problems as a result of different philosophies? Are those dittrenccs known 
and appreciated by the critical parties? 

e. Project Activities 

( 1 ) Discuss all major activities^ undertaken by the project since funding. Provide a detailed account- 
ing of the programs and service's provided by the project. If the project has been supported by 
grant funds in previous, years, very briefly describe project status during each of them. Provide 
detail only for the current year unless a specific issue requires further elaboration. 

(2) Do current activities deviate frpm what w£^$ outlined in the grant contract? Are these changes 
appropriate? * ^ 

(3) To wTiat extent is the piroject operational in relation to the work schedule included in the grant 
contract? If the project is behind schedule, what prpblems have been encountered? How have 
they been overcome?^ 

(4) Are there any constraints on p^ject operations which have not already been discussed? 

(5 ) Are there any problems which have not already been^jlentified? 

f. Linkages 

( 1 ) Describe which criminal justice and other agencies relate to the project and their role. 

(2) Have any problems developed in this ^j:«f? Wjjat has beeh done to resolve the problems? 

g. Client Flow and Characteristic^. , ^ 

(1) Discuss the number of clients served since the beginning of the grant period. Provide adequate 

detail by type of service and/or outcome as appropriate. . . 

(.2) Indicate, to the extent possible, relevant client characteristics. 

h. Feedback about the Project v 

( 1 ) If possible discuss attitudes about the project on the part of the community, clients, user 
agencies, etc. , ' ^ 

(2) Indicate what problems, if any, have been identified, how this affects project operations* and 
what should be or is being done about them. " . 

Not all of these subsections will be appropriate for all projeycts. Nof will the information always be avail- 
able for the second report. It is assumed that the final report will be more comprehensive. However, to the 
extent they are applicable, each of the^abov^ areas should be examined. It is anticipated' that subsections 
a,* b, and e will always be included in the second and find reports. ^ 

D, TECHNICAL AND FISCAL (include in all reports except final): 
Discuss separately, if appropriate, the following items: 

L Quarterly Progress Reports ^ 

Progress Reports Received " . ' 

- 1st. . 2nd,.^.^^ — ^3rd_ 4th 5th 

(Date) (Date) (Date) (Date) (Date) 

a. Have quarterly reports been received onetime? 

b. Do they provide sufficient detail? 

2. Accounting and Record Keeping ' , * 

a. What role is actually performed by the office of the financial officer? 

b. Are review procedures for encumbrances and expenditures adequate? 



c. Who is keeping the books? 

d. Have alli'pcrtinent parties, including the project director, been briefed oafiscal policy? 

e. Do records provide sufficient detail? 

f. Has soiree documentation been retained? 

g. Have any problems developed due to conflicting fiscal policies among OCIP, th% applicant, and the 
implementing agency? * . 

3* Fiscal Reports and Revisions 

a. Have 201s been submitted each require moiith ind on time? Are they up to date? 

b. Have budgetary changes within the discretion of the subgrantee been reported on 201s? 

c. Have budgetary revisions been made in accorclaiice with OCJP fiscal policy? . . 

d. Is a budgetary revision required? * " 

e. Have expenditures been made in accordance with the grant contract? . , 

4* Cash Flow 

Date I ■- ' — — ' — 

Grant Grant Match ^ Encumbrances 

Award ' Expenditure . . 

$ $ . ' • $J $— ■ . 

a^ If this rate of expenditure is projected to the end of the grant project, will all grant funds be 
expendcjd? If *not, are there larger items of expenditure expected at a later time within the grant 
period? Examme figure? by s^ubcategory if appropriate. Plict figures in perspccUve— check date*of 
201 report. 

b. If a surplus is projected, should the contract duration period be extended or should thp contract 
amount be reduced through amendment, thereby freeing up the money? , ^ 

^ 5. Procurement ' 

a. Btas equipment been purchased in accordance with the grant coojract? 

b. Has any equipment been ordered during .the last 90 days of the project? (For projects under 73 

manual) ^ ' 

c. Have procurement instructions pertaining to competitive bids ahd contractual instruments been 

followed? . 

d. Will equipment and supplies be delivered within the proper time frame? 

e. Have any problems developed with regard to the disposition of equipment at the end of the grant 
period? 

. EVALUATION METHODOLOGY . ^ 

1. Methodology for Current Report (include in all reports) : * ^ . 

a. Discuss type and frequency of contacts made in the preparation of this report; list dates of major 
contacts as appropriate. . • i 

b. List principle individuals interviewed in the preparation of this report; omit names and simply 
specify affiliation if identity could breach confideflfiality or have negative repercussions. 

2. Methodology for Long-term Evaluation (if different): 

a. Describe the evaluation design. , 

b. Identify the evaluator (if other than Board staff) and discuss various roles, as approj^riatc. 

c. Is the evaluation being implemented in a timely manner? 

d. Have any difficulties arisen between the evaluator and project staff? 

e. Are there any problems with respect to evaluation which have not already been disctissed? What is 
being done about them? 

It is anticipated that this section will focus on evaluation methodology and implementation. Actual Infor- 
mation obtained as a result of the evaluation will be incorporated— as appropriate— in the previous 
sections. 
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2. Ekampb of a Monitoring System Using 
Specif i«ci Performance Factors 

Michigan's SPA administered monitoring system 
is based on a specified set of performance indicators, 
a part of which is assigned to each program category 
funded by the SPA. The performance indicators, 
called "inspection factors," are designated for each 
program category in the annual plan. Exhibit 1 is a 
complete list of inspection factors. Occasionally a 
grants administrator at the SPA will change the 
factors for a particular project. The factors are 
assigned when grants are awarded and the regional 
planning unit personnel as well as project directors 
are notified. The projects are then monitored by 
regional planning staff on schedules appropriate 
for each project. They make subjective assessments 
to the SPA. The RPU monitor indicates pass or fail 



for each inspection factor and then writes a short 
paragraph about each factor. Exhibit 2 is an example 
of a typical set of inspection factors assigned to a 
project. 

Project directors also submit reports to the SPA. 
Their qyarterly progress reports are narrative repqrts 
describing project development, work and accom- 
plishments. Exhibit 3 is a copy of this report. Th^se 
quarteriy reports, as yell as the RPU monitcffJng 
i*eports, are sent to the.grants administration divi- 
sion of the SPA. They are the means used by the 
grants administrators to keep informed about pro- 
ject's progress. In order to check the reliabUity of 
the information project directors and RPU personnel 
are sending to the SPA, the Grants Administration 
Division sends teams/of two people to inspect a few 
projects in each regional planning disttict 
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Inspection Factors Dictionary 



INSPECTION FACTORS * 

01 Examine staff selection procedure. . ' ^ 

02 Verify employment of project staff. 

03 Inspcctadequacy of staff work stations. 

04 Review staff training procedures. 

05 Examine method of staff activity reporting. ^ 

06 Examine trainee attendance reports. 

07 Examine traine^ selee;tion and recruitment procedure. * 

08 Examine training curriculum and materials. , ' " 

09 Examine traming schedule. > . - 

10 inspect training sit?s. ^ 

11 Determine percent of trainccis completing course. , . 

12 Examme for compliance with pertinent reflations, 

13 Ascertain availability of professional services. 

14 Examine method of consultant selection.^ ' ^ 

15 Examine consultant contracts: - 

16 Examine consultant progress reports. , 

17vv Examine procedure fo^documenting equipment use. . « * * . ' 

18 Verify acquisition and installation of equipment. 

19 Examine intake arid output;«eportiirig procedures. i 

20 Review procedures for comjpiling statistical data. ^ ^ 

21 Examine methods and procedures for follow-up. 

22 Inspect procedure and documentation evaluating project effectivene^. 

23 Examine project referral procedures. 

24 Verify implementation of project. 

25 Determine if project is on schediilc, . ^ . ^ 

26 Verify establishment of qualified advisoVy council or group. 

27 Examine reporting procedures of Advisory Council or group. ^ ^ 

28 . Examine project facility for adequacy. - 

29 Obtain subjective evaluation from project personnel. 

30 Obtain subjective evaluation from affected agencies. " ^ 

31 Obtain subjective evaluation from trainees. 
^32 . ^ Determine degree of inter-agency participation. 

3 33 * Examine subgr^antee method of monitoring project. 

34 Ascertain policy and procedural changes as a result of project, 

35 Examine documentation and/or method of selecting target population. 

36 Examine administrative organization. , 

37 Examine extent of project^s effect geographically. 

38 Examine method for making project adjustments. 

39 Inspect documentation for completion of major tasks specified in application. 

40 Qualifications of project staff are consistent with application. 

41 Qualifications of project staff are not consistent with application. 

42 Exception to the qualifications of staff has been corrected. 

43 * Documentation identif> mg ihe target population is adequate. . ^ 

44 Documentation identifying the target population is riot adequate. 

45 Exception to the identification of target population has been corrected. 
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mSPBCTION FACTORS 



Procedures for documenting project goal ittainment are adequate. f 

Procedures for documenting project goal attainment are not adequate. 

Exception to tht procedures for documenting project goals conccted. 

Procedures for documenting goal attainment of target population are adequate. 

Procedures for documenting goal attainment of target population are not adequ 

Exception to procedures for documenting target population goals corrected. 

Completion of major projects tasks identified in application is on schedule. 

Completion of major project tasks identified in application is behind schedule. 

Exception to completion of mapr project tasks has been corrected. 

'Documentation of goal completion is adequate. 

Documentation of goal completion is not iidequate. 

Exception to the documentation of goal completion correctied. " " . 

Examine efforts to recruit minority applicants. 

Staff qualifications are consistent with application. 

Procedures for identifying target population are adequate. 

Administrative organization is adequate. 

Documentation identifying the target population is adeqtiate. 

Documentation of project goal completion is adequate. 

Documentation of goal attainment of target population is adequate. 

Procedures for evaluating.effectiveness of project are adequate. 

Degree of inter-agency cooperation is adequate. * 
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STATE Of MICHK5AN 
OFFICE 6f CRIMINAL JUSTICE PtOGRAMS 
Sacofid FI«Qr, Ltvris Cats luiMiiii 
Unsinf, Mkfiigan 4t91S 



Project Inspodiofi Report 



PlMncial Dlr«ctef 

Financa Diroctor 
City« Michlfan 



Captain * 

County Sheriff's Dapartment 
City* Michigan 



1:30 p.m. 
4/3/7 4 



Shariff '$ Papartnwnt 



$27>340 



IntiMcter Ntm« 



SiibcrantM >«rs«fln«l t«ntactH 

Captain 



Sheriff 



Subrvnt Iff*** 0«M 

7/1/73 to 
6/30/74 



Typt of ln»f«ctl«h 

Program 



OOOOO-l 



inm 



% Camp. 
40 



[Q (ntartm 
Q Final 



Suhgrantaa 

County 



County 

Polfca Cadat Program 



tNaffCTiON rACTont ^ « 
EXAMINE TRAINEE ATTENDANCE REPORTS 



aaa Fall 

ED 



EXAMINE TRAINING SCHEDULE 



EXAMINE PROCEDURE FOR DOCUMENTING EQUIPMENT USE 



ED 



DETERMINE IF PROJECT IS ON SCh<EDULE 



0n 



OBTAIN SUBJECTIVE EVALUATION FROM AFFECBED AGENCIES 



ED 



OBTAIN SUBJECTIVE EVALUATION FROM TRAINEES 



□ □ 



□ □ 



B«Md on tht abovt Itttad factors * this projt«ct httt 



\ 



Copy 

□ mff 

[]] INSf ECTOR 

PIIOJ^CT OmECTOR 

□ H^E INSfCCTION 
Q fOST IN»PECriON 



[XXXXX | fat— cl 

j „ ] Nmd wrth condition* (tttjittachmtnt) 
[ I r«il«d (Furthar inttructtont forthcomint) 



REQIONAl omCCTOR 
[Jl PRr iNSPtCTION 
Q POST INSPECTION 
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STATE OF MICHIGAN 
OFFICE OF CRIMINAL JUSTICE PIKXSRAMS 
S^coTNi Flo«r* Uwls Cats ■ulMIng 
UnttnCf Mkhlgan 4t91i 
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SUBGRANT QUARTERLY 
PROGRESS REPORT 


SuN(r«rrCM (nam* aitd «Mr«Mj 


Subcr«nt Amount Contrvt Num^r 

OCJP 
LEAA 


(If p*ft Hf Q^artir Entrfmg 

[ 3 March 31 [ ]Junt30 t ]S#ptember30 
t ] DtcembtrSl 




guartarly frogras* 


Rfport Numbar 

0 


^tr^od Of Fundrng 



Oat« •! Last rimiiKlat Itof 

0 


Wft SiibtTHtt**! 


fadtial fund* Eupcmitti t« Oata 


P«fcant 0» f re|tct Actkvifltt C«mpl«t«a 











For Ipstructions Sm N«xt Pag6 



Quartarly Projtct Report Narrative 



Submitted Bvt 
Projict Director^ 
Title ^ 



Signature. 



Prepared By (if other than project director): 

Name . 

Tittn 



Data 



Signature. 



on* 
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PROCEDURES: SUBGRANT QUARTERLY PROGRESS REp6RT 

L Quarterly Progress Report Requirement 

A quarterly progress report is required for all active action and discretionary projects. The Subgrant 
Quarterly Progress Report form must be completed and submitted at the end of each calendar quarter 
and must be received not later than 20 days after the end of the quarter, 
II, Distribution 

A. Action Subgrants 

* * Submit the original and one copy of the Subgrant Quarterly Progress Rcpbrtjo: 

Office of Criminal Justice Programs / 
Second Floor, Lewis Cass Building / 
JLansing, Michigan 48913 ^ / 

B. Discretionary Subgrants 

Submit the original and one copy to:* Submit two copies to; * 

LE AA Region V Office Office of Criminjil Justice Programs 

0*Hare Office Center, Rm. 121 Second Floor, I^fewis Cass Building 

3166 DCS Plaines Avenue Lansing, Michigan 48913 

DesPIaines, Illinois 60018 

NOTE: LEAA considers submission of the OCJ? Subgran^ Quarterly Progress Report form an 

acceptable alternative to its own progress report procedures aiid form. 
\lL Form Completion / ^ ^ 

A* Provide general project identification and statistical information requested in the heading portion of 

the form. Several requested items require explanation. 

Quarterly Progress Report Number— Indkait whether this/is the 1st, 2nd, 3rd, etc. quarterly progress 
report submitted * ^ ^ 

Period of Funding—Enitx the subgrant effective date and terminatiop date. The period of funding is 
indicated as such on the Subgrant Contract. 

Federal Funds Expended to Da/f— Enter the total Fede/al funds expended and encumbered as of the 

. last Financial Report* ^ / > , 

Percent of Project Activities Completed— Indkdit the Actual percentage of project activity completed, 
as described in the project schedule in the appro vedjkjjplicatiori, regardless of timfe elapsed. 
B. The quarterly report narrative must include inforttiation which accurately describes the state of 
project development, work and accomplkhments to date. Pay specific attention to project phases or 
stages completed. Be concrete and specific concerning accomplishments, e.g„ number of people 
trained, volume of correctional services provided, extent of equipment usage. If the Subgrant Con* 
' tract special conditions indicate reporting requirements, be sure these are met. Attach numbered 
continuation pages as needed. The quarterly report narrative should answer the following kinds of 
questions: 

L Is the piroject proceeding according to the project time schedule which was previously determined? 
. If the project is not on schedule, explain the cause of delay and what measures have been or will 
be faken to correct the difficulty. * 

2. What appropriate data or reports are available to document project progress? For example: train- 
ing schedules, agendas, equipment purchase data, lists of clients served, equipment utilization logs, 
lists of training session attendees, credentials of staff hired. Attach this information to the report* 

3. What unexpected or new factors have affected the development and implementation of the 
project? In wh&t way has or should, project activities be altered to adjust to these factors? 

^'^ 4. How arc activities related to the evaluation component of the project progressing? Discuss any 

written or oral evaluations (or tests) which h^fe been completed. Will the project accomplish 
the original objectives, or should the project Scope and objectives be reconsidered? 

5. Will the project be 'completed within original (or revised) time and resource limits? If not, what 
modifications arc needed? Wijl an exfenslon be required? 

6. Arc there any facts or findings to date that might have a bearing on other ongoing or planned 
projects of similar nature? 
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PtrOGRESS RETORT (Contlnutd) 



Grant 
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Actlvtty ' 


- Projtcttd Amount 
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NOTE;: PA|RT (--Must ba complated prior to cKsbursemanl to Subgranlaa,, 
^ P^m ll-»To ba complatad by Subgrantee. 
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